Understanding preconsumed space in a hard disk volume

Question

NOTE: I'm working specifically with VHDs, but I'm assuming similar rules govern physical hard disks. Please correct me if I'm wrong in that assumption, but even if I am, it's a moot point for my particular use case.

CONTEXT

OS: Windows 7 (x64) Ultimate

I'm attempting to create single-volume VHDs that have free space of a given size, e.g. if I want exactly 2 GB (2,147,483,648 bytes) free after all initializing and formatting are completed, I figured out I have to create the VHD as 2,092 MB (2,193,620,992 bytes).

I am using MBR, which I've discovered will consume 3,076 KB (3,149,824 bytes) upon initialization, regardless of VHD size - I'm assuming to store the partition / volume information?

However, in my 2 GB example, I would then create a single NTFS volume using the full available space, and once the drive is available, you can see that some space is already used - 41,980 KB (42,987,520 bytes) to be exact.

QUESTIONS

1. What is this space being used for? Is there any reason it has to be used, or can it be reclaimed for normal data storage?
   
   

2. Is there a calculation I can use to know how much space will be consumed after the NTFS volume creation?
   
   I'm going to want VHDs of different sizes, and would like to be able to pre-determine this lost space, rather than using trial and error. I've attempted creating VHDs of different sizes and comparing the values of the used space, trying to find either a consistent percent-of-original used or a fixed-amount + percent-of-original. I haven't been able to determine a consistent pattern.
   
   

3. Would my end goal be easier to accomplish via GPT instead of MBR? I haven't bothered experimenting with it.
   
   

4. Likewise, would it be easier with a format other than NTFS?

Answer 1

What is this space being used for? Is there any reason it HAS to be used, or can it be reclaimed for normal data storage?

It's the MFT (Master File Table). By default 12.5% of the drive is reserved for it, but it could also grow larger if you have tons of files.

This is the main metadata file, in this case it's the list of files stored on your drive. Each file and directory needs at least one entry recording it's id, the name, MAC times (Modification, Access, Creation) and some other stuff.

Please note that $MFT is not the only metadata file in NTFS. There are other elements, such as $MFTMirr, $LogFile and so on. If you are curious, my MSc thesis contains a bit more information about this.

Is there a calculation I can use to know how much space will be consumed after the NTFS volume creation?

You could probably come up with a very complicated one but it would work only at the immediate moment of file system creation. As soon as the OS starts working on the drive, something more may get written on the file system.

Would my end goal be easier to accomplish via GPT instead of MBR?

GPT is a way of storing the partition table (OK I am quite simplifying here). The file system format (NTFS) does not change.

Likewise, would it be easier with a format other than NTFS?

Not really. All file systems need some metadata. There are archiving solutions like cpio that basically "smash" files together but they do not work well as file systems.

Also, you are using Windows which is a pretty clueless operating system when it comes to working with file systems (and a lot of other tasks). Therefore your choices are limited.