Fastest way to count zero bytes on a partition

Question

My root and home partitions are LUKS-encrypted. I wish to have discard/TRIM enabled on them (I'm aware of security implications). If my understanding is correct, this should result in discarded areas being zeros on a raw disk device and gibberish on cryptsetup-mapped device.

I've made some configuration changes, ran fstrim and now I want to verify if discard is working. To do that, I want to count how many bytes on my partition are zeros. On a trimmed disk I'd expect that:

count of 0 bytes ÷ total partition size ≈ percentage of free space

On a non-trimmed, encrypted partition: (assuming all bytes were written at least once)

count of 0 bytes ≈ total partition size ÷ 256

Inspired by How to gather byte occurrence statistics in binary file?, I've tried this approach:

sudo pv /dev/disk/by-label/ESP | od -vtu1 -An -w1 | grep -Fx '   0' | wc -l

but results were disappointing: processing a 500 MB partition took almost 3 minutes. That's 3 MB/s, while my SSD can reach sequential reads of 360 MB/s. I've also noticed that one of my CPU cores was running at 100%, while others were idle, so I guess this could benefit from parallelization. (Impact of pv is negligible, almost no difference compared to cat)

What's the fastest way to count 0 bytes on a partition?

Answer 1

If you are able to access the raw disk (without encryption) you could do:

dd if=/dev/sda bs=1M count=100 2>/dev/null | tr -d '\0' | wc -c

That will read 100Mega byte from the disk, remove all the zeros and count the number of bytes remaining.

I do not know if that is the fastest method but it should be quite fast assuming that there are more zeros than data.

The command above will read 100 times 1Mega byte or 100*2^20 (104857600) bytes (if they are available). Try with a small number (10 or 100) to test speed and how well the command works. To read the whole disk just remove the count option, let the side of the disk limit the command.

Extracting the two values that matter (full disk byte count and number of non-zero bytes) the command become more complex:

$ time { { { sudo dd if=/dev/sdX bs=100M 2> >(grep -oP '[0-9]+(?=\s*bytes)' 1>&2 ) | tr -d '\0' | wc -c; } 2>&1; } | { read -d '' a b; echo "$b/$a"; }; }

Which will print the two values as: non-zero/disk, like:

766568020/999292928

calculate 1-766568020/999292928 to find the percentage of zero bytes:

$ bc <<<'(1-766568020/999292928)*100'
23.28895776994831289400