4

I have a 1TB hard disk with 512-byte sectors that I have been using through a USB disk enclosure for a good many years. However, the enclosure has now died.

When I transferred it to a new enclosure I found that Windows doesn't read it - it just shows up as a 'Local Disk'.

From various research, I have found that the cause is that the original USB enclosure was translating the 512-byte sectors on the disk and presenting them to the outside world as 4k sectors, instead. The new enclosure is not doing this, which is why the disk cannot be read.

I have found a question which sounds like a similar problem (How to correct 512-byte sector MBR on a 4096-byte sector disk?), but (a) from my reading, this sounds like the opposite of my situation; and (b) it doesn't describe the actual steps needed to fix the problem (just a general approach).


The theory

Aside from attempting to find another enclosure that does the same kind of translation (which feels like just kicking the can down the road, and an unnecessary expense) the solution, as far as I can glean, is to fix the values in the MBR so that they reflect the real 512-byte disk geometry rather than the previously perceived 4k geometry.

As the partition table sizes are all defined in terms of numbers of sectors, I will need to convert the existing values, which specify the size in 4k sectors (i.e. ByteOffset = N x 4096) so that they instead contain the equivalent value of N for 512-byte sectors (i.e. ByteOffset = N x 512). In other words, I need to multiply each of these values by 8.

Q1: Am I correct - is that all I need to do, or is there more complexity to it than that?


The practice

Assuming the above solution is correct, what I am really asking for is whether there is a tool or a set of clear instructions that anyone can provide so that I am able to make these changes to the disk. The less-risky, the better! (For example, tools for direct editing the MBR fields by entering decimal numbers would be preferable to having to deal with hex-values directly on disk.)

Q2: What tool(s) should I use for this (based on the above criteria)?

Q3: What fields/offsets do I need to edit? (The disk has a single partition which fills the disk.)

Note that I have access to the following systems: Windows XP, Windows 10, Ubuntu 15.4. My preference would be for a Windows solution, but a Linux solution would be fine if that is necessary - I am comfortable using the command-line.

Also, I have done byte-level manual disk editing tasks in the past, so this is a possibility if there really is no better way to resolve this.


Update

The external drive is a Dynamode 3.5" USB 2.0 enclosure, like this one.

As requested, here is the output from the Linux fdisk command.

root@buttons:~# fdisk -l /dev/sdb

Disk /dev/sdb: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000210df

Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1  *     2048 1953523711 1953521664 931.5G  7 HPFS/NTFS/exFAT

root@buttons:~# fdisk -l /dev/sdb1

Disk /dev/sdb1: 931.5 GiB, 1000203091968 bytes, 1953521664 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x2052474d

Device      Boot      Start        End    Sectors   Size Id Type
/dev/sdb1p1 ?       6579571 1924427647 1917848077 914.5G 70 DiskSecure Multi-Boot
/dev/sdb1p2 ?    1953251627 3771827541 1818575915 867.2G 43 unknown
/dev/sdb1p3 ?     225735265  225735274         10     5K 72 unknown
/dev/sdb1p4      2642411520 2642463409      51890  25.3M  0 Empty

Partition table entries are not in disk order.
12
  • 1
    I don't think you can fix this with just modifying the MBR fields. The NTFS filesystem itself on that disk is also internally calculating with 4K sectors and you have no way of adjusting that. Rescue your files (you already know how) and re-partition. It is a pain in the ***, but that is the only sure and safe way of recovering your data.
    – Tonny
    Commented Oct 28, 2020 at 14:18
  • 'Ubuntu 15.4', or better 15.04, is quite rather old and unsupported: Vivid Vervet end of life was on February 4, 2016. You may want to try (maybe on a USB pen) a more recent version. BTW it may result useful to edit your post adding the specifications of the old usb box. Maybe it is somehow still possible to emulate the hardware, install the firmware on the vritual device and give the hdd to the emulated machine with the aim to extract all as soon as it possible. Next format it again (and buy a replacement, old hardware is prone to break).
    – Hastur
    Commented Nov 2, 2020 at 12:46
  • @Hastur Yeah - I know it's old, but none of the update channels work (it looks like they've gone offline), so it's not trivial to upgrade and I don't use it enough to invest the time in figuring out how to work around this. It shouldn't matter for the purposes of this question, though. Interesting your point about emulating, but unless I'm missing something, this will result in a copy+reformat+copy back, which seems like a more complicated version of my answer, below. Nevertheless, I'll try and get the specs for the old enclosure, in case that helps anyone.
    – HappyDog
    Commented Nov 2, 2020 at 14:04
  • @HappyDog The idea of formatting is to avoid any unnecessary layers. In general you should be sure in every case that you get the correct data, now and in the future. Since it seems like it is not possible to know if the firmware has introduced anything beyond block translation, you (or anyone in a similar situation) should find the same hardware / firmware mount the disk (physically), get the data, etc. If the latter is not possible, emulate the hardware (if possible) and install the original firmware. If you can't even do that, go ahead and save the data as you can (continue).
    – Hastur
    Commented Nov 2, 2020 at 14:32
  • The problems with unnecessary layers (even if in principle handled correctly) are the general slowdown and the lack of optimization by the system. Furthermore you would have (1) the HDD firmware layer, (2) the one of the box (possibly emulated) and the one (3) of the operating system. If the central layer (2) is not longer or in any case necessary, it is better to remove it: how? Recover data, reformat HDD with new hardware and put data back. But mind that if the box hardware is already broken, the HDD hardware may also be at the limit. Backup, always.Twice. ;-)
    – Hastur
    Commented Nov 2, 2020 at 14:40

3 Answers 3

1

This is not a direct answer to my question, as it doesn't apply a proper fix to the disk (which is what I am most interested in).

However, from a practical stand-point it solves the problem, in a rather slow and roundabout way.

I have found a free tool called TestDisk by CG Security, which seems to run on pretty much any operating system. The tool can be instructed to read the disk using arbitrary sector sizes, which means we can override the physical sector size reported by the drive and recover the data from it.

How to use

Having downloaded and installed this command-line utility, run it, make a decision about how any log files should be created, and then:

  1. Select the disk from the list. Make sure you select the physical Disk rather than the logical Drive.
  2. Select the Intel partition type.
  3. Select the Geometry option and change Sector Size from 512 to 4096. This will allow TestDisk to read the disk properly.
  4. Select Advanced which takes you to the file utilities, and - if there is more than one partition - select the partition you are interested in. You should also at this stage note-down the partition sizes, if you plan to rebuild the disk later.
  5. Select List to view the files on the disk.
  6. From here you can browse the file system, select individual files or groups of files, and copy them to another location (i.e. to a different disk).
  7. To copy the whole disk, from the root directory, select a to select all files, then capital C to copy the selected files, then browse to the destination disk and press capital C again to copy all the files across.
  8. Repeat if there are multiple partitions.

You have now made a backup of the whole disk! You can therefore use the standard operating system tools to wipe and repartition the original disk as per the previous layout (this will automatically use the correct geometry now that there is no disk enclosure translating the sectors) and copy the files back to complete the rebuild.

Note that the above instructions only restore the disk partitions and file structures but this may not be sufficient if you also need boot sectors, use non-NTFS drives, have files that need to be in particular disk locations (rare) or use other more sophisticated gubbins (e.g. alternative data streams). All of these issues may or may not be resolvable using this tool, but not with the above instructions.


(Note that I don't really want to have to accept this as the answer, as it is actually side-stepping my question rather than resolving it. However, I am noting it here as an alternative approach.)

2
  • If the OS was reading/writing to the drive in real 4096-byte sectors instead of 512-byte sectors, then very likely the metadata of the filesystem, such as the MFT if you formatted this NTFS under Windows, is also expecting that. There's no way to fix that without completely translating/rewriting the MFT. That's all over the disk. You would need a utility that is specifically designed for this purpose; I don't know of any. Perhaps there is one and someone will respond with it, but I'm betting you're going to have to do what's in this answer.
    – LawrenceC
    Commented Oct 28, 2020 at 15:37
  • @LawrenceC You might be right about that, in which case going in that direction (updating the MBR/MFT/etc. to use 512-byte sectors) may turn out to be practically impossible. But what about going the other way, so we update the disk to be marked as having 4096-byte sectors? If this were possible then everything should be consistent, no? This seems to be what TestDisk does in software, but is there a way to make this permanent so that the OS, and therefore all other software on the machine, can read the disk properly?
    – HappyDog
    Commented Nov 8, 2020 at 16:15
1

It is possible to do this by creating partitions of the same size and in the same location using fdisk on Linux. You need to translate the start and end values based on the difference in sector sizes. It looks like the current values match the size of the disk (did the enclosure map the partition table too?) so you only need to fix the NTFS boot sector.

Filesystem metadata for FAT and NTFS depends on the disk sector size so you'll have to modify it. If the NTFS cluster size is smaller than 4K then it will not be possible to change in the 512 to 4096 direction.

If it's NTFS you can edit the boot sector (at the start of the partition) with hexedit on Linux changing the following little-endian values (instructions assume a 4K cluster size):

  • 0x0B (Bytes per sector) 0x0200 (512) <-> 0x1000 (4096)
  • 0x0D (Sectors per cluster) 0x08 (8 * 512 = 4096 byte clusters) <-> 0x01 (1 * 4096 = 4096 byte clusters)
  • 0x1C (Hidden sectors) The number of sectors preceding the partition
  • 0x28 (Total sectors) The partition size in sectors minus 1

Reference: NTFS Partition Boot Sector

If you were booting Windows 10 from this disk then you would need to preserve the disk and partition GUIDs. The EFI System Partition (FAT32) must be at least 33MB (normally 100MB) for a disk with 512 byte sectors and 257MB (normally 260MB) for a disk with 4096 byte sectors (I recommend just recreating it because it contains few files and is easy to copy).

2
  • Thanks for the suggestion. The disk is wholly readable if I use TestDisk to simply change the sector size from 512 to 4096. I don't know what that does internally, but my assumption was that it is simply overriding one of the disk parameters. What I haven't figured out is how to make that change 'stick'. I am not sure whether this answer resolves that (as it appears to require edits to two linked parameters). Either way, this is not a bootable disk, so the final paragraph, whilst interesting, is not applicable in this case.
    – HappyDog
    Commented Sep 5, 2021 at 18:55
  • My answer should make the change permanent; I've added a link to the Wikipedia content that describes the boot sector.
    – Simon
    Commented Sep 6, 2021 at 20:02
0

You may want to consider trying to change the logical sector size on the disk interface to match the 4K sector size that was managed through the defunct USB interface:

hdparm --set-sector-size 4096 /dev/sdX    # for an appropriate value of X

You'll need a reasonably recent version of hdparm to find this option. I don't know if Ubuntu 15.4 is recent enough (and if not, you can maybe download a Live version).

Obligatory warning: I don't have a disk that is mangled in the same way for me to test this command.

6
  • Update: failed, with an error message (run as root): hdparm --set-sector-size 4096 /dev/sd resulted in The running kernel lacks CONFIG_IDE_TASK_IOCTL support for this device. READ_LOG_EXT(0,0) failed: Invalid argument READ_LOG_EXT(SECTOR_CONFIGURATION) failed: Unknown error -2. Thoughts?
    – HappyDog
    Commented Nov 6, 2020 at 18:39
  • Sorry, no other ideas. It sounds like it needs a newer kernel than the one you're using Commented Nov 6, 2020 at 18:48
  • Kernel version is Linux 5.4.0-52-generic, which is the latest LTS version, and Ubuntu is 20.04, which is also the latest LTS version. There are later versions of each, but I'm hesitant to upgrade to a non-LTS distro or to mix-and-match kernal and OS.
    – HappyDog
    Commented Nov 6, 2020 at 19:07
  • That odd error just likely means your USB-HD3.5 enclosure isn't passing the commands. So if you want to go down this road you better try to figure out what usb bridge it employs (the smartmontools guys are pretty crafty in figuring out per-controller workarounds) or just temporarily connect your drive to some "native" internal SATA port. But if your disk isn't 4Kn, going through sector translation twice seems just a waste, and I'd rather just rebuild the partition table.
    – mirh
    Commented Jan 13, 2021 at 19:17
  • I've finally found some time to do a bit more digging with this. It turns out that you were right and that the above error was related to the USB drive enclosure. I put the drive inside the machine via a direct SATA connection to the motherboard, and that particular error message resolved itself...
    – HappyDog
    Commented Aug 2, 2021 at 10:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .