The SD card is – next to the power supply – a critical additional component of the Raspberry Pi. Monitoring it’s health is really important to ensure a smooth operation of your Raspberry Pi operating system, and a good user experience. This article will show you several ways how to check and monitor the health of your microSD card.First, I’ll give an in-depth overview of how memory cards work, so you can understand the possibilities and limitations of checking the health state of your SD card. Then I’ll explain how to protect your microSD card by reducing common problems Raspberry Pi users experience. We’ll also go into the best microSD card brands for Raspberry Pi we recommend. If you want, you can also skip down further, to just get the Linux commands to check the current microSD card state.
Basics: Inside the microSD card
The microSD card contains a flash memory chip (on the left of the picture), and a micro-controller (on the right of the picture, usually ARM-based).
Flash memory stores information by “trapping” electrons, which are “injected” using high voltage through a non-conductor into a so-called floating gate(**). The electrons are thus part of a transistor that may or may not allow a connected current to flow, depending on the charge of the floating gate. Theoretically, they cannot flow away, because the floating gate is electrically isolated. This means that the information is retained even after the current supply is switched off.
The information is always read between source (S) and drain (D). Electrons introduced into the floating gate increase e.g. the threshold voltage of the transistor, starting from which current would flow. The transistor then blocks at a normal read voltage (does not conduct).
For programming the floating gate, significantly higher electrical voltages (e.g. 10 V) are used than for the normal read operation (e.g. 3.3 V). For this purpose, additionally the control gate (V1/V2/V3) is used.
For erasing, the electrons are driven out of the floating gate again by applying a high negative voltage via the control gate.
NAND flash components used in microSD cards group the individual memory transistors into pages, and several of the pages into blocks. A page has between 512 and 8192 bytes, a block can contain up to 256 pages (thus a total of 2048 kB with 8kB page size).
Writing (for a logical “1”) can be done bitwise or at least byte / wordwise. Erasing (for a logical “0”) can only be done blockwise. Information that has not been changed must still be programmed in again.
Flash memories have a limited lifetime due to programming and erasing, which is given in erase cycles
The reason for the limited lifetime is damage to the insulating oxide layer, which protects the floating gate from charge leakage, caused by the high voltages. As soon as this layer becomes conductive, no more information can be held in the memory cell.
Aside: Multi-level cell memory cells
Initially, there were only two charge states (1 bit of information) per memory cell. Now, thanks to several floating gates per transistor, multi-level cell memory cells store different charge states and thus several bits per memory transistor. The transistor then conducts the applied current differently, which is evaluated during readout.
On the one hand, this makes it possible to increase the density of the memory cells significantly, but on the other hand, readout is slower and the memory cells react much more sensitively with bit errors to charge losses. With single-level cells, 100,000 to 1,000,000 write-erase cycles are possible, with TLCs (triple-level cells with three bits per memory cell) approx. 1000 write-erase cycles.
This is the reason that industrial SD cards usually have lower memory densities, and use SLC (single level cells), for better data integrity.
The task of the controller is to manage the flash, and in particular to perform wear leveling and read error correction. The performance and longevity of the microSD card depends decisively on the algorithms used in the controller.
Flash memory cannot be rewritten as often as required due to damage to the insulating oxide layer of the floating gates as described above. To avoid damage to individual areas that are used particularly frequently, the controller varies the physical allocation to the blocks that can be addressed logically by the file system (= wear leveling).
This variation of the physical allocation is also the reason that write-testing the SD card (by writing & reading with bad block tools, etc.) will actually not identify the real bad blocks & allow you to avoid them on the operating system / file system level! Only the flash memory controller inside the microSD card knows which block is written to / read from at any given moment, and as discussed this can change over time.
Defective blocks (bad blocks) are already present in brand-new flash memory. These defective blocks are marked in a special area of the flash memory.
Error correction information for individual blocks is also managed so that read errors can be corrected by checksums. The controller adds blocks with clustered read errors to the list of bad blocks, and shifts the actual physical allocation of the logical block.
The microSD card typically has – depending on the manufacturer – about 10% spare capacity to exchange the bad blocks with good “reserve blocks”.
Dirty little secrets: Flash memory problems
Deletion is only blockwise
Data can only be erased block by block. Erasing stresses the memory cells, and shortens their lifetime – new bad blocks are created.
Defective blocks from the factory
Flash memories are already shipped with defective blocks. In the course of operation, further defective blocks (bad blocks) are added. The controller therefore tries to write / erase blocks as evenly distributed as possible by wear-leveling.
MLC and TLC particularly sensitive
Multilevel cell memory cells (MLCs) reduce the number of erase cycles and thus long-term reliability.
A phenomenon not yet mentioned by me, but particularly perfidious, is Read Disturb. Even when only reading from the card, it can – just by reading – cause neighboring memory cells in the same block to change their programming. The probability of this happening increases sharply after a few 100,000 reads.
To avoid Read Disturb the controller therefore logs the number of accesses to a block in order to copy it in one piece to a new location when a threshold is exceeded, and to delete the original block. After that, the block can be re-used again.
All these are things a controller has to compensate for in order to pretend to us that it is a “perfect memory card” on the outside, while it looks anything but perfect on the inside!
Last but not least, bits written by X-rays could be erased unintentionally. Here, only an X-ray-safe design of the card can ensure that the data remains intact.
Manufacturer & selection of a good card
Both micro-controller and flash device and the finished microSD card can come from different manufacturers – the Panasonic SD card in the example photo has Samsung flash, and the controller was made in Japan.
NAND flash devices are produced by four manufacturers:
- Samsung <- market leader
- IM Flash Technologies (Micron Technology & Intel joint venture)
- Hynix in cooperation with Numonyx
Toshiba and Samsung produce the majority of all chips.
SanDisk and Toshiba have a joint venture for flash manufacturing. However, in 2009, SanDisk transferred the rights to the fabs to Toshiba to become a fabless flash memory manufacturer. Memory development is still done together. SanDisk and Toshiba, along with Matshushita, are the founders of the SD standard, introduced in 1999. The microSD standard was also created by SanDisk.
We have been relying on the SanDisk brand for a long time, and have had very good experiences with it so far.
Samsung memory cards are also frequently recommended. As the market leader in the NAND flash sector, Samsung can match all components of the SD card perfectly to each other, and has all the necessary information for a solid product.
TL;DR NOT recommended
We had a bad experience with the reliability of 128 GB Kingston cards in a critical project. Kingston does not have its own fabs, and buys excess capacity from other flash manufacturers. As a result, consistent performance cannot be guaranteed.
The following article by Bunny Huang is also interesting in this context.
Toshiba, as the #2 in the worldwide DRAM market, also sells its own memory card products. SanDisk and Toshiba, together with Matshushita, are the founders of the SD standard, introduced in 1999. We have no experience with Toshiba cards so far.
Transcend / Silicon Power
We have also used Transcend and Silicon Power cards, especially in the low-cost range. The cards are basically good, but we have seen higher returns / defects with Transcend than with SanDisk. For critical projects, I would therefore rather recommend SanDisk / Samsung.
Choosing a good memory card
In order to select a good memory card, the first thing to do is to choose the manufacturer. We recommend Samsung or SanDisk, and possibly Toshiba, for the shortlist.
Next, the memory class should be considered. This class indicates the fastest write speed block by block. It is not the write speed for scattered random access writes, which is more meaningful in real-world applications.
A higher class is more expensive, but recommended due to the significant performance increase. We usually ship Class 10 cards.
In this overview on eLinux.org a suitable memory card can then be checked for compatibility with the Raspberry Pi. Important note: the Raspberry Pi firmware has been modified several times for better compatibility with memory cards and to avoid data corruption.
You should order from trustworthy sources – for example, we get our SanDisk microSD cards directly from Raspberry Pi Trading, and reputable, large German distributors. On Amazon, be sure to order directly from Amazon, not from a Marketplace retailer.
Certain information (cid, csd, date, manfid, oemid, serial) of the microSD card can be read with Linux to check that you have received the correct manufacturer. Examples with a Transcend card:
cd /sys/class/mmc_host/mmc?/mmc?:* echo "man:$(cat manfid) oem:$(cat oemid) name:$(cat name) hwrev:$(cat hwrev) fwrev:$(cat fwrev)"
man:0x000074 oem:0x4a60 name:USD hwrev:0x1 fwrev:0x0
echo "serial:$(cat serial) mdt:$(cat date)"
The Manufacturer ID (manfid) is assigned by SD-3C LLC, as well as the OEM / Application ID (oemid). The oemid identifies the OEM of the card and/or the content of the card.
The product name (name) is 5 characters long (ASCII). hwrev is the hardware revision, and fwrev the firmware revision. Together they are the product revision (hwrev.fwrev).
The serial is the serial number of the microSD card, it is a 32bit field that should be read as unsigned integer.
The mdt (Manufacturing Date) indicates when the card was manufactured – year and month.
Stable man/oem combinations of a brand indicate a well-controlled and consistent supply chain. For example, SanDisk has the following combination for all entries in the eLinux.org Wiki: man:0x000003 oem:0x5344. Other brands, such as Transcend, vary their suppliers, so consistent quality can no longer be guaranteed.
Avoid data corruption during operation
The microSD card is the Raspberry Pi’s “hard drive”. With a normal Linux configuration, it is treated exactly the same way a hard drive would – both read and write accesses occur. Linux logs information to many log files, and among other things, updates file access times. Data is thus frequently changed and overwritten. This is something microSD cards were never designed for in the first place.
Tip #1: Shutdown Raspberry Pi properly & use a good power supply
You wouldn’t just unplug a Windows computer either. With Linux computers, and especially the microSD card-supported Raspberry Pi’s, it’s only a matter of time before data corruption occurs during powercuts.
Linux has read-write caches to speed up file operations. When you just unplug power from the Pi, you may lose the information in the write caches that has not yet been written to the microSD.
As mentioned earlier, the microSD controller is actively shifting data at runtime at runtime for wear-leveling, and avoiding read disturb. This happens without the Linux system’s knowledge (“transparent”). And depending on the manufacturer, more or less conservatively. Also here – especially during write operations! – data damage and losses can occur.
So you should make sure that, after shutting your Pi down, the ACT-LED of the Raspberry Pi stops blinking, and only then pull the power.
A bad power supply can also lead to problems due to brownouts (undervoltage). If in doubt, use the power supply recommended / supplied by Raspberry Pi Trading.
Tip #2 Read-Only System with Overlays
A read-only system does not protect the SD card from read disturb, and thus from aging due to the necessary erase cycles. However, this happens much slower than when actively writing.
A read-only file system also has other advantages, for example fewer file system checks at startup.
Especially embedded systems – i.e. systems that have been developed for a specific purpose and are then installed in an application (e.g. digital signage) do not necessarily need the possibility to constantly install new software and to keep precise logs on site. The logs can be sent over the network to a central server (delayed if necessary), and for system updates the system can be put into maintenance mode to apply the updates.
Here is some information from the Debian project on which Raspbian is based: https://wiki.debian.org/ReadonlyRoot.
Using RAM disk overlays (in tmpfs), a system can still write log files, or make changes to files. The RAM disk overlays could be periodically synchronized with specially designated partitions on the SD card. Care must be taken with the RAM disk so that it does not overflow (due to the log files), and there is of course less RAM memory available for normal activities.
We use this technique with our product Anonymebox, where it must be assumed that users simply unplug it from the socket.
Are you looking for a partner to realized a solid read-only solution for your embedded Raspberry Pi project? We (as pi3g e.K.) offer consulting & development services, for hardware and software.
Tip #3 Reduction of write operations
Especially with older kernels, Linux updates the access time for each file access, according to the POSIX standard. This means that every read operation automatically results in a write operation. There is the possibility to explicitly set noatime in /etc/fstab if it is not already set (Raspbian seems to set it automatically).
Disable system log using mask:
systemctl mask systemd-journald.service
Created symlink from /etc/systemd/system/systemd-journald.service to /dev/null.
sudo systemctl mask rsyslog.service
Created symlink from /etc/systemd/system/rsyslog.service to /dev/null.
Other logs created by other applications, and the consequences of disabling these services (for example, certain services no longer starting) should of course be examined in detail.
If there is not enough RAM, Linux moves individual RAM areas into a SWAP file. (Consider using a Raspberry Pi 4 with enough RAM, for example the Pi 4 / 4 GB or the Pi 4 / 8 GB for your use case.) With the following command the status of the SWAP file can be checked:
sudo systemctl status dphys-swapfile
● dphys-swapfile.service - LSB: Autogenerate and use a swap file Loaded: loaded (/etc/init.d/dphys-swapfile) Active: active (exited) since Sat 2017-07-01 19:11:57 UTC; 8min ago Process: 498 ExecStart=/etc/init.d/dphys-swapfile start (code=exited, status=0/SUCCESS)
The following command can be used to disable SWAP:
sudo systemctl disable dphys-swapfile
microSD cards in continuous use put the long-term reliability of Raspberry Pi-based systems to a severe test.
By using the right power supply, read-only systems with overlays, consistent reduction of write operations, and selecting a good SD card by a well-known brand, stability can be increased in the long run.
(*) Note: the Raspberry Pi 3, 3B+ and 4 can boot from USB media, or over network (Ethernet), without microSD card.
Especially for a boot over network, it is still recommended to install a microSD card with a special firmware, because a timeout bug in the boot code prevents a stable boot (Raspberry Pi 3B).
The explanations above about the structure of flash memory also apply to USB sticks, since flash memory is also installed in them.
(**) Note: an increasingly used alternative to floating gates are charge trapping flash memories, the functional principle remains the same. The charge trapping flash cell enables higher memory densities.