Manual NTFS recovery sanity check

Question

I need someone to please sanity check my manual NTFS recovery process so far because it's lead me to a brick wall.

Background:

• I had a 1TB NTFS external drive (WD Elements) which contained mostly photos.
• Somehow, it's corrupted and appears as a raw disk on Windows.
• It appears on a Linux system in the /dev/disk/by-path (and by-id, by-uuid etc.) directories and appears as /dev/sdb.
• EaseUS is able to find (nearly?) all of my photos on there with a quick scan (not the heavy lifting deep scan)
• EaseUS finds about 70GB of files (mostly photos).
• I think the NTFS records are corrupted i.e. it's not a mechanical failure.
• I'd like to attempt recovery myself for fun and profit.
• I don't have another drive large enough to make a complete image of the corrupted drive.

I need to parse the NTFS MFT $File records because:

• I'd like to get the original file names and directory structure back.
• If an image is not written in contiguous clusters I won't successfully recover it just by looking for image file signatures.

The plan is:

1. Image a portion of the corrupted disk.
2. Parse it to identify and read MFT $File records.
3. Use the $File records (and specifically the $Data attribute thereof) to determine the data runs of each file.
4. Knowing the data runs for a file, I can pick out the bytes of a file from the image I created using ddrescue.
5. Rinse and repeat until I'm done.

Firstly - does that sound like a reasonable plan?

What I've done:

1. Found a bunch of $File records
2. Parsed one to get the data runs
3. Read the raw bytes at the location specified by the data run.

Specifically:

1. Used ddrescue to image 100GB (starting at 0) of the corrupted disk.
   1. I imagine that nearly all of the actual data I need is written within the first 100GB since the total volume of interesting data is 70GB. I can repeat this whole process over subsequent 100GB portions if necessary.
   2. The command I used to image the first 100GB was ddrescue /dev/sdb ~/mydisk.img ~/mydisk.map --size=100G.
   3. ddrescue did encounter I/O errors and reported that it only recovered about 99.57%.
   4. The start of the image (first 20MB or so) seems empty so this might be the reason for the drive failure. I'm ignoring that for now.
2. Read over the 100GB image and located all instances of the ASCII string "FILE" which denotes the start of a $File record in the MFT.
   1. This has also triggered on false positives such as the word "PROFILE" in the middle of an arbitrary file.
   2. Therefore I'm only considering results where the distance between one occurrence of "FILE" and the next is <= 1024 bytes since that's the maximum MFT record size. If there's 3MB between one occurrence of "FILE" and the next then it's unlikely to be a $File record.
3. Iterate over presumed $File records (size <= 1024 bytes) and extracted $Data attributes.
4. Parsed it for data runs (ignoring the discussion about resident vs non-resident attributes which I think I understand but isn't part of my question).

So I went through the above process and selected one of the $File records and identified its data runs. Here's the $Data attribute (header and contents):

80 00 00 00 48 00 00 00  01 00 00 00 00 00 01 00
00 00 00 00 00 00 00 00  FA 03 00 00 00 00 00 00
40 00 00 00 00 00 00 00  00 B0 3F 00 00 00 00 00
F4 AC 3F 00 00 00 00 00  F4 AC 3F 00 00 00 00 00
32 FB 03 ED C8 11 00 00  FF FF FF FF 82 79 47 11

The data run details are the first half of the last row, just before the FF FF FF FF end of attribute marker:

• length byte: 32
• cluster run number: FB 03 (little endian) = 1019 clusters in the run
• cluster start number: ED C8 11 = 1165549 is the first cluster of the run
• the next 00 indicates that there are no more runs.

Now, considering there are 512 bytes per sector and 128 sectors per cluster, I read (1019 * 128 * 512) bytes from the 100GB image starting at (1165549 * 128 * 512).

Unfortunately that left me with a 66.8MB file of mostly 0x00 although towards the end there was some data. I'm pretty sure I've got something wrong and I just found some data by accident (although I do happen to end on a JPG end-of-file marker (DD F9).

Is my approach to this whole task reasonable and I've just made a small error somewhere?

Or have I misunderstood something fundamental about NTFS and this is completely the wrong idea?

Answer 1

Firstly - does that sound like a reasonable plan?

No. I mean I didn't look in depth at you method for decoding runs but, start clusters are relative to start of file system ( = start of volume / partition when we talk NTFS). So any MFT entry that points to a cluster number, points to cluster number relative to start of partition, not relative to your arbitrary 100 MB part of the volume.

Also, MFT is 'self referencing', so first thing to try is to find start of MFT from which you can then derive all MFT entries. If you can not find start of MFT try locating the mirror of the MFT instead.

So, to properly decode a MFT entry and get the data it is referencing we need:

• Offset to start of partition.
• Correct cluster size

So if we decode start cluster we can do: (cluster no. * sectors / cluster) + offset partition

How did you determine 128 sectors per cluster? this does not seem right at all! See defaults here: https://www.disktuna.com/default-cluster-sizes-for-fat-exfat-and-ntfs/.

Answer 2

It's unprofessional to expose a dammaged disc to the read load you described. The first requirement is to duplicate that drive to a stable healthy one to avoid the loss of additional sectors and to have no unreadable sectors anymore. The fact that the unreadable sectors could not be copied is sad but the key is to avoid you or a recovery programm to handle read errors.

Your recovery attempts may very well have already caused additional dammage on that drive.

As metadata structures and data are not necessarily laid out in a linear manner you cannot compensate the lack of owning at least one drive that is big enough by subsequently reading 100GB slices somewhere into free drive storage elsewhere. You need random access. Unfortunately in your case once a structure points into your next 100GB slice you have to empty your 100GB storage area.

If you are already parsing structures in your preferred programming language there is no need to run unix commands like dd for the copy job. ddrescue is needed just once at the beginning of your recovery attempt.

If you just want to learn NTFS internals that's fine and a cool thing to do but you can learn that already on storage like USB sticks. Big drives are not necessary.

Please think about for what reason Easeus did not recover your desired metadata like file names and folder structures.

Answer 3

I thought drives were linear starting at byte 0 of sector 0 of cluster 0 and going all the way up to the end of the physical drive.

Yes and no. Storage addresses are linear, starting with byte 0. Sector addresses are linear as well.

Cluster addresses are linear and starting with cluster number zero but only relative to their respective partition. There is no disc-wide cluster labeling only a partition wide cluster labeling starting from zero.

Are you saying that, if I use a tool like ddrescue to read bytes 0 - 1000000, I shouldn't expect to have read the NTFS metadata which describes the layout?

The NTFS metadata involves not only the boot sector but the master file table and other files as well. When reading 1 MByte from the start of the drive you will only read a fraction of the master file table provided that it is located there. Its position is not fixed and is likely to change upon defragmentation.

Use the strategy to experiment and learn about your file system here: https://forum.cgsecurity.org/phpBB3/viewtopic.php?p=31304#p31304

When trying to decrypt NTFS you are so much better knowing the intended outcome already. It is a kind of deciphering process then like a "known plaintext attack" where you know the whole plaintext.

Somebody just downvoted my answers. If that was you please explain the reason. Is there an error somewhere? I'd like to know! Thank you.