65
\$\begingroup\$

I can't find any reliable information about this. I don't have the full specification of the SD/MMC Card hardware.

Is it true? My high-level application doesn't need to be concerned with wear leveling when working with these cards?

EDIT

Could someone confirm that wear leveling is guaranteed by the SD specification? I want to be sure, because it looks like most vendors do it, but it is not required by the specification.

\$\endgroup\$
1
  • 5
    \$\begingroup\$ Not all flash media manufacturers invest in wear leveling algorithms. Wear leveling is more prevalent in SSDs where the manufacturers lead the market because of algorithms. \$\endgroup\$
    – Chetan Bhargava
    Commented Feb 14, 2013 at 5:15

7 Answers 7

Reset to default
52
\$\begingroup\$

I work for a company that used to be a member of the SD association, we are familiar with the 2.0 (SDHC) spec. The SD card spec has NO entry for wear leveling. That is completely dependent on the SD manufacturer to handle that if they so choose. We have seen that some likely do, while others very much do not (beware the super cheap knock-off SD cards). SDXC may have changed that to include wear leveling, but I am unsure of that. Unfortunately the only way to really show that is to get your hands on the official spec. You can find it online most likely, but the SD association really wants you to pay for it.

As a side note, taking a 2GB card and writing it beginning to end over and over again averages about 10TB before the card is dead and no longer is writable. Also, SD cards will not let you know when data is bad, i.e. wont return an I/O error like a PC harddrive will. This might not be an issue for embedded designs as 10TB is a LOT of data, but it could be a factor for someone.

\$\endgroup\$
7
  • 2
    \$\begingroup\$ I assume that even though it isn't in the spec, companies still want to do it to keep their reputation good. While super cheap knock-offs don't care about reputation, they sell to people who look to go cheap on everything. Does this sound reasonable? \$\endgroup\$
    – Kellenjb
    Commented Mar 5, 2012 at 22:25
  • 1
    \$\begingroup\$ @Kellenjb It is a very reasonable assumption, and I personally would agree with you. However, because it is not in the spec, and it's very unlikely to gain access to any particular manufacturer's SD controller designs, it can't really be proven. We have done a lot of testing over the years, some major brands do perform much better than others. It could be differences in internal algorithms or complete lack of them. \$\endgroup\$
    – Kris Bahnsen
    Commented Mar 5, 2012 at 22:36
  • 4
    \$\begingroup\$ Cheap knockoffs aren't going to care even if it IS in the spec. It's not immediately visible to the customer, it COULD be missing. We're talking about people who sell lead plugs as ICs. I haven't heard of a name-brand card that didn't do wear leveling, but I suppose it's possible. \$\endgroup\$
    – darron
    Commented Mar 5, 2012 at 22:40
  • 3
    \$\begingroup\$ @fredbasset, way to zombie the thread ;) Its been a while since I did any SD card work. However, IIRC SD cards themselves will physically return whatever they feel like. Some of the newer ones combined with decent drivers may be able to detect an IO error. However may of the older SD cards will just return bad data that was stored in NAND and call it good. Though, if a write is not able to complete, that will return an IO error. \$\endgroup\$
    – Kris Bahnsen
    Commented Jun 13, 2014 at 5:50
  • 1
    \$\begingroup\$ @fredbasset Generally, if linux gets bad data, it has a fit. The company I work for has implemented a checksumming process directly on the block device that goes through a software layer. Data recovery and errors are now able to be reported to the kernel through that setup. Raw NAND is its own beast, but from what I've experienced it is more robust in terms of handling faults. Check out UBI and UBIFS, its basically JFFS3, really robust and offers some speedups. \$\endgroup\$
    – Kris Bahnsen
    Commented Jun 26, 2014 at 23:37
27
\$\begingroup\$

It's true! MicroSD cards contain a NAND flash chip bonded to an (ARM) microcontroller encapsulated in some black plastic. http://www.bunniestudios.com/blog/?p=898 explains.

At the end of the follow-up post http://www.bunniestudios.com/blog/?p=918 Bunnie posits integrating the controller probably costs less than testing the flash ahead of time.

Quoting SanDisk's SD card product manual: "1.9 Wear Leveling. Wear leveling is an intrinsic part of the erase pooling functionality of the SD Card, using NAND memory." You can read the whole thing in the datasheet for a SanDisk brand card.

\$\endgroup\$
7
  • 1
    \$\begingroup\$ Bunnie Huang is definitely credible enough for me! \$\endgroup\$
    – vicatcu
    Commented Mar 5, 2012 at 18:09
  • \$\begingroup\$ That's good to know, but I'll wait for someone that can enlighten this question based on the formal specification of this technology. \$\endgroup\$
    – ivarec
    Commented Mar 5, 2012 at 19:07
  • 2
    \$\begingroup\$ Digikey showed me this useless SanDisk brochure when I tried to look up a datasheet. The Wintec and Swissbit datasheets were better, and both mentioned wear leveling. Can you link to that Digikey-hosted datasheet? \$\endgroup\$
    – Kevin Vermeer
    Commented Mar 6, 2012 at 18:36
  • \$\begingroup\$ Those are way better than the old SanDisk one. \$\endgroup\$
    – joeforker
    Commented Mar 6, 2012 at 19:12
  • \$\begingroup\$ The Wintec wasn't great, but the new SanDisk one is the best of the bunch! Thanks for the edit. \$\endgroup\$
    – Kevin Vermeer
    Commented Mar 6, 2012 at 20:09
21
\$\begingroup\$

Yes, SD/MMC cards have controllers that do wear leveling. If they didn't, you could destroy one in a matter of minutes with the wrong write patterns.

That's actually a problem for some embedded projects. There's absolutely no way (apparently) to know what sectors might be wear leveled at any time, so a power cycle at the wrong time can destroy data anywhere on the card, no matter where you THINK you're writing. (don't ask how I know :) )

SD cards must be used with a system that guarantees a clean system shutdown (or at least that writes are allowed to complete), or data loss will (eventually) result.

EDIT

The problem is that the wear leveling process is entirely hidden. ANY sector on the disk could be moved at any time (swapped with the page written), and if power was to fail in the middle of that process that random sector could get corrupted.

While there ARE reasonably safe ways to implement this move, it's not in any spec so you can't trust that the card will do it. You could test one card, have it work, then the manufacturer could change the implementation without changing the part number and you're screwed.

From testing, my SD cards' controller does NOT do this in a safe manner at all.

I may look into a "high reliability" SD card I saw advertised specifically for power failure tolerance... but then you have to trust the manufacturer to do that correctly, and I don't. I really want direct control over page erases. I'm still trying to figure this one out.

\$\endgroup\$
10
  • 4
    \$\begingroup\$ See my posted answer. Since wear leveling algorithms are up to the manufacturer, they might not be up to snuff. A good wear leveling algorithm will move the data first, then mark that data good, then garbage collect the original data. This might be an issue on some cards, maybe not others. SD is a very ugly design when you get to the bottom of it. \$\endgroup\$
    – Kris Bahnsen
    Commented Mar 5, 2012 at 22:16
  • 2
    \$\begingroup\$ Yeah, it's a crap standard as far as I'm concerned. It's very annoying that such a ubiquitous memory storage standard would be so inherently unreliable. \$\endgroup\$
    – darron
    Commented Mar 5, 2012 at 22:43
  • 2
    \$\begingroup\$ @jpc: Maybe I'll do a blog entry then. I'm getting the impression it's not considered often. I didn't realize the implications myself until it was too late. I fought with this, talked with my SD card manufacturer, etc. No solution other than to minimize write time. I now write to NAND flash, and copy to SD once a day as quickly as I can. There ARE SD cards designed to be "resistant" to random power failures, but I'm not sure even these are completely trustworthy. \$\endgroup\$
    – darron
    Commented Mar 6, 2012 at 15:38
  • 2
    \$\begingroup\$ @fred basset Oddly, the specs on industrial cards seems to have changed since this answer was written. I was seeing microSD cards with guarantees on writes (journaled, I think)... now, they make no such claim. Talking to a manufacturer, they're using supercaps now... but only the compact flash cards have room for them. Doing things safely reduces the performance too much. They're saying to just somehow provide enough backup power to complete a write. Given how complex modern ARM SBCs are... it's really hard to do if the vendor's reference design doesn't do it. None that I know of do. \$\endgroup\$
    – darron
    Commented Apr 27, 2013 at 20:24
  • 2
    \$\begingroup\$ @freq basset I'm using UBI & UBIFS. It seems to work well... I've had a lot of systems running for years now with no problems. \$\endgroup\$
    – darron
    Commented May 24, 2014 at 16:32
12
\$\begingroup\$

Any type of SD card using any type of conventional NAND flash memory is going to have to use some type of sector virtualization, since no conventional NAND flash device can support erasure of individual 512-byte sectors, and no conventional NAND flash device of significant size would be able to yield performance that was within an order of magnitude of being even marginally acceptable if every attempt to write a sector required the device to copy all the sectors in that sector's erase block (even to RAM), then erase the block and write all the sectors back. Most sector-virtualization techniques are inherently somewhat wear-leveling. I would expect the biggest issue of variance between quality devices and knock-offs will be the extent to which a device actively tries to even out the leveling between blocks, versus simply using pseudo-random block allocation and hoping that will yield acceptably close-to-uniform results. In practice, I would expect that even random/hope-for-the-best allocation would be adequate in most cases.

\$\endgroup\$
12
  • \$\begingroup\$ It very well might be that the "wear leveling" some manufacturers implement is caused by the NAND interface itself and block to sector/sector to block virtualizations. \$\endgroup\$
    – Kris Bahnsen
    Commented Mar 6, 2012 at 0:09
  • 1
    \$\begingroup\$ @KrisBahnsen: I would expect that manufacturers which claim wear leveling will actively monitor the relative wear of different blocks and age of the data therein, and if blocks with long-lived data are found to be in blocks with much less wear than average, the data from those blocks will be moved into blocks that had more wear (so as to, most likely, minimize future wear on those blocks). This could improve by a factor of 10 the useful life of a device that's 95% filled with data that never changes, while the remaining 5% of space is churned frequently. \$\endgroup\$
    – supercat
    Commented Mar 6, 2012 at 21:25
  • \$\begingroup\$ I agree, if they are advertising it I imagine they would do some kind of real wear-leveling. I am a bit negative on the whole subject which I do apologize for; I've been bitten a few times by the quirks of SD. \$\endgroup\$
    – Kris Bahnsen
    Commented Mar 6, 2012 at 22:02
  • 1
    \$\begingroup\$ @KrisBahnsen: The biggest problem I think is a virtualization model which assumes a linear bunch of consecutively-numbered 512-byte sectors. It may have been convenient to interface with DOS, but it doesn't really fit well with the hardware that exists, nor with what the host software really wants. Open up block numbers to 64 bits and allow them to be arbitrarily non-consecutive, and then arrange to have files always stored in logically-consecutive blocks. To delete a file, delete its range of blocks. \$\endgroup\$
    – supercat
    Commented Mar 6, 2012 at 22:56
  • 2
    \$\begingroup\$ This is possibly the best answer. I suspect this is how all of them do it. If you think about it, there is no way to do deterministic wear leveling with any kind of truly sophisticated algorithm because the wear history itself would have to be stored somewhere on the same card and that part would fail first. Randomized leveling is the only practical possibility. I am not convinced SSD's are any smarter. They may simply have more replacement buffers, though it is certainly within possibility that they store very coarse (and hence infrequently updated) wear information. \$\endgroup\$
    – Nimrod
    Commented Jan 2, 2016 at 9:08
9
\$\begingroup\$

Sandisk have a white paper that explains the wear levelling logic in their cards, and goes on to give estimates of the card's life under a number of scenarios. Executive summary: unless you are hammering the card non-stop, it will last decades.

\$\endgroup\$
2
2
\$\begingroup\$

Its interesting to note that despite this a lot of devices do corrupt SD and microSD cards especially high density ones if the battery is low or the phone crashes/shuts down/etc. I suspect that the problem is inadequate regulation of the voltage supply as this is well known on some cards (cough Adt /cough) to result in the phenomenon of an unreadable card on certain external readers but works fine on the micro variety supplied by some computer shops.

I am in the process of recovering a card at the moment with this fault, strangely enough most of the data is recoverable but some sectors are not though this changes on each attempt. Could the wear leveling itself be at fault? (yes tried multiple readers, same fault!)

\$\endgroup\$
1
\$\begingroup\$

Also had some success "nuking" zombie cards, ie ones which won't complete a format or are read-only. Only works on a small percentage but they are much more sensitive than most "official" guidelines to the effect.

A test card thus treated lasted a full four months before failing again, if it hadn't been for the device it was used in supplying noisy voltage the lifetime might have been longer.

\$\endgroup\$
3
  • 1
    \$\begingroup\$ By "nuking" you mean putting it in a microwave? At what wattage and how long? \$\endgroup\$
    – 0x6d64
    Commented Jan 30, 2017 at 12:31
  • \$\begingroup\$ Uh, no :-) Also why do that when a glow discharge (RF) would have a similar effect and be a lot less hazardous than the method I used? Turns out that uSD cards are a lot more sensitive due to the lack of a metal casing on one side. \$\endgroup\$
    – Conundrum5642Z
    Commented Jan 31, 2017 at 10:36
  • \$\begingroup\$ I can't confirm or deny that nuking refers to using a "gadget" 5642+HVPS for legal reasons. However it did work repeatedly, and the technique is now known as I published it on H-a-D etc. \$\endgroup\$
    – Conundrum
    Commented Feb 1, 2018 at 7:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.