SD card lost filesystem - what happened?

Question

I'm attempting to setup a network with hundreds of fanless (ARM)PCs. Each will have its filesystem on a 32G sd card. The image I'm working with uses Debian 6.x and the card is formatted with EXT2 as one large partition.

Last night one of the test machines started throwing errors as follows:

sd 0:0:0:0: [sda] Device not ready
sd 0:0:0:0: [sda]  Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda]  Sense Key : 0x2 [current]
sd 0:0:0:0: [sda]  ASC=0x3a ASCQ=0x0
sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 2d 78 86 00 00 08 00
end_request: I/O error, dev sda, sector 2979974
EXT2-fs (sda2): previous I/O error to superblock detected

sd 0:0:0:0: [sda] Device not ready
sd 0:0:0:0: [sda]  Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda]  Sense Key : 0x2 [current]
sd 0:0:0:0: [sda]  ASC=0x3a ASCQ=0x0
sd 0:0:0:0: [sda] CDB: cdb[0]=0x2a: 2a 00 00 01 78 86 00 00 08 00
end_request: I/O error, dev sda, sector 96390
EXT2-fs (sda2): error: read_block_bitmap: Cannot read block bitmap - block_group = 11, block_bitmap = 360448

Does this mean the media is bad? Or decaying somehow? I'm going to try reformatting it and see if it happens again.

Is there a better way to setup this card? Different file system?

EDIT: Possible related post here.

Answer 1

I can only tell from my own experience with SD cards. I once had a problem with an sdcard used in my smartphone. It seemed corrupt and I couldn't use it anymore. It was FAT32 formatted. On my OSX laptop I had similar problems. On a Windows XP machine I could access the card and make a backup. After reformatting it, no problems at all. I found it strange that only under Windows I had no problems. The card was never before formatted in a Windows machine, although maybe in the factory?

In my opinion sdcards (what I read about them) are not reliable for running an OS. You probably found a problematic card or a faulty formatted card. Reformat it, mark it so next time you know if this card had a problem before, and keep a log of problems with these cards. Why do you choose sdcards?

Answer 2

SD cards do not have any kind of intelligent wear-levelling on them so it is entirely possible for you to wear out blocks or groups of blocks that store important filesystem information making that area of the card essentially unusable.

When you reinstall you are not checking for bad blocks in the filesystem, you are simply copying an image over the card in a linear fashion and generally ignoring whether or not the write operation actually wrote the correct data or if the block is able to hold the data.

If you were able to format the card and do a bad block scan then you would likely find some areas that were unusable.

The reason a lot of devices allow you to use SD cards for this is completely clear. SD cards are dirt cheap, available everywhere in a variety of sizes, and almost universally supported by small embedded processors. It's not Kingstons fault (nor any other SD card manufacturer) that people have chosen to use the cards in a fashion that is almost guaranteed to send the card to an early grave. If your card dies, you just reinstall and carry on.

The price point for these ARM based PC devices is too low to expect any mission-critical level of reliability. Generally if you don't want the SD card to die then you need to move the filesystem off to a more reliable type of device. Some Raspberry Pi owners have gone so far as to move the root filesystem to a USB hard disk drive for reliability (hard drives do not suffer in the same way that SD cards do) or even net-boot the devices.

If you really are using the cards a lot you may want to look into sites for SD card based ARM devices

• Raspberry Pi: Prevent SD-Card Corruption
• Howto optimize Linux – SD card as rootfs
• How can I prevent my Pi's SD card from getting corrupted so often? from Stackexchanges very own Raspberry Pi site