3

That seems completely backwards. (The block size on this NFS share is 512KiB, causing 20% waste, but that's a different issue.)

$ df -BM --output=used,target /Database/backups
    Used Mounted on
3842164M /Database/backups

$ du -cd0 -BM --apparent-size /Database/backups
3841946M        /Database/backups
3841946M        total

$ du -cd0 -BM /Database/backups
4631230M        /Database/backups
4631230M        total
2
  • 2
    output of df -BM --output=fstype,used,target of the underlying filesystem (not the NFS mount) might be useful
    – user10489
    Commented Jun 23 at 13:05
  • 1
    It might be worth explaining a bit why you find this to be completely backwards? Is it the other way around on other filesystems? Also, that 512 kB block size sounds quite large, is that NFS server some "special" system?
    – ilkkachu
    Commented Jun 23 at 16:10

1 Answer 1

2
+200

The man page for du right in the short description says it estimates file space usage. There are many filesystem features that can make its output inaccurate.

For sparse files, block size would always be smaller than apparent size and apparent size would grossly overestimate storage size.

For nearly all other cases, apparent size would not take into account block granularity storage of files and metadata blocks and thus would underestimate storage size of a file.

The only things I can think of that would make du overestimate number of blocks would be specific to the underlying storage filesystem and would include:

  • block packing, storing multiple files in a single block.
  • filesystem dedup and copy on write

Both of these would cause du to overestimate the number of blocks used, but only the former would not also increase apparent size.

So for du apparent size to be closer than du block size to df output, maybe the underlying filesystem is doing block packing making block counts useless for size estimates. Knowing which filesystem (df -T on the server) would help determine if this is likely.

The other possibility would be if du is basing its block estimate on larger blocks (e.g., 8k) than what is actually being used (eg., 512b), but this seems unlikely as it would be something the filesystem could report correctly.

3
  • /Database/backups is mounted nfs4. I don't know what the underlying PowerScale NAS filesystem is.
    – RonJohn
    Commented Jun 28 at 14:01
  • Wouldn't surprise me if the powerscale is doing both block packing and dedup.
    – user10489
    Commented Jun 30 at 14:30
  • Yeah, I got that same bad feeling. The problem is that du --apparent-size on that NAS can be S-L-O-W, and df is untrustworthy.
    – RonJohn
    Commented Jun 30 at 14:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .