How to avoid dd or badblocks erasing hard drives' spare sectors?

Question

/!\ Look for Update 2 down there, the question is half-answered now /!\

I sometimes completely wipe my hard drives (not SSDs); being what they are, dd and badblocks -w do wipe everything blindly.

The thing is, they are also writing hard drives' spare sectors, which is bad since they are all brand new. So if something do happen during the erasure process (power loss, write errors, etc), that would kill a very useful sector... for nothing.

So the question is: How to avoid dd or badblocks erasing hard drives' spare sectors?

---

Assumptions:

• dd is known to blindly erase everything that goes through it.
• badblocks is known to be able to access unmapped blocks.

Update 1:

dd definitely can't access spare sectors if they didn't replaced a faulty sector (obviously). This is due to how dd was designed (comment by Kamil Maciorowski):

The logical sector numbers available to dd are mapped to numbered physical sectors and the corresponding numbers are initially equal. Spare physical sectors initially have no logical numbers assigned, so dd cannot reach them.

Update 2:

Being mostly a sysadmin -and not being a "close to low-level" coder (C)-, can someone take a look at badblocks' code and tell if it does access spare sectors? To be more precise, does badblocks also take into account spare sectors inside its block reading/writing?

http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/tree/misc/badblocks.c

On another side/way to get the answer; I also know that Theodore Ts'o, the badblocks' maintainer, has an account here on Super User. If he reads this update 2, is it possible to have a direct answer about that?

Answer 1

Neither dd nor badblocks is able to access disk hardware spare sectors. The main reason dd and badblocks can't access those spare sectors is that the authors of those utils don't know how, and the proprietary nature of modern hard disk hardware and SMART software makes it prohibitively difficult to find out.

---

This Q seems to be based on a confusion between OS level file systems and disk vendor's hardware level SMART and remapping routines. A historical view might clarify things...

Early PC hardware vendors tested disks before shipping, and included lists of bad blocks on the disk label or an included printout. The admin who installed the drive would then perhaps use that list of bad blocks when formatting the drive. Early formatting programs would prompt the user for lists of bad blocks, which the user would type in by hand. If the drive had been around a while, the admin might recheck the drive blocks. Anyway the point of those bad block lists was to tell the file system not to use those blocks. The badblocks util seems to be designed with that sort of early hard disk in mind.

Newer hard disks include remapping routines, as well as diagnostic routines such as SMART, the both of which amount to something of a dedicated computer within the hard disk that checks for bad blocks and even remaps bad blocks, automatically. Wonderful functionality, but what sucks about vendor remapping is that all the implementations of it are proprietary and closed source -- so users aren't sure what it's really doing, or failing to do in there. A disk's remapping/diagnostic firmware might be buggy, there might be secret places on the disk colonized by spies or black hat hackers, etc. Since it's disk level remapping, (not OS level), it can inefficiently cause more head thrashing and wear, (if some often used file or region is in a spot with remapped sectors).

Answer 2

Per your request, I have taken a look at badblocks.c and I can testify that it it is a very simple program that does not use any advanced disk function and does not work on the disk-controller level.

All it does is use the Linux standard open function to open the drive as a raw device (meaning as a virtual file that includes the entire device), while applying the O_DIRECT flag to do I/O directly to the disk without going through the Linux memory buffers.

It then retrieves the number of blocks in the raw device using the ioctl function, a number that includes only publicly visible blocks/sectors (no spare sectors), then iterates writing and reading a pattern to each block/sector, using the Linux standard functions of write and read, reporting an error if any error code is returned or the data is not equal.

The open/read/write functions date from the very beginning of Linux, when disks had a much simpler structure and no spare sectors. Much more advanced functions would be needed to access any internal disk data such as the spare sectors which are unavailable through casual use.

As a curiosity, the comment at the beginning of badblocks.c says :

* This file is based on the minix file system programs fsck and mkfs
* written and copyrighted by Linus Torvalds <[email protected]>

where Minix pre-dated Linux (history link) and whose shortcomings actually motivated Linus Torvalds to write the first version of Linux. This is just to get an impression on how ancient and unsophisticated is this program. It certainly does not touch the spare sectors.

I have also looked at dd.c which is a much more sophisticated program that attaches the disk-driver. I am not familiar with the disk-driver interface, but it seems to me that even if the spare sectors could be modified via that interface, no programmer in his right mind would do that. My reasons are :

1. dd can write a disk-image to and from a file, and that image does not contain the spare sectors (except when mapped), which probably means that dd does not touch unmapped spare sectors.
2. The spare sectors are used to remap bad sectors. Monkeying with them will mean that bad sectors will become unmarked and new data copied to such bad sectors may be corrupted and lost without notice.
3. The spare sectors mapping is unique to its disk, as two different disks can almost never have exactly the same defects, so copying or clearing that mapping makes no sense.