Recommended zfs properties for blockchain data mountpoints

Question

Blockchain based technologies require large datasets to work. It makes sense to put them on a separate volume/dataset in case the filesystem supports them. I'm using ZFS and was wondering whether there're any properties recommended to set on the dataset which only contains blockchain data. Is it well compressable (if yes, with with algorithm)? Is it deduplicable? What's the drawback during reading (e.g. is old data accessed often and therefore needs to be decompressed constantly)?

I'm assuming that they all have similar data structures, but if not let's focus on what bitcoin stores in ~/.bitcoin.

I'm using ZFS 7.x on Ubuntu 17.10.

Answer 1

In bytes, the on-disk format for the Bitcoin blockchain is made up mostly of signatures and hashes.

In ZFS or any other storage system, there are three main techniques for byte efficiency:

1. Compression: Hashes and signatures mostly look like random numbers, so it will be difficult to get much gain here.
2. Deduplication: Dedup requires exact block matches to work, but I don’t see any reason why those would exist in the blockchain — I think every write will be unique.
3. Snapshotting / cloning: This only helps if you need multiple writable copies of the blockchain.

So for the most part, my answer is that ZFS (and other file and block storage systems) can’t really help... unless you store multiple copies of the blockchain.

If you have multiple copies for some reason, you can probably get massive benefits from dedup, since IIUC every node must eventually store the exact same blockchain. You could use one NFS server using ZFS, and mount clones of the same filesystem everywhere you need a copy of the blockchain. That’d also make creating new copies much faster, since you could immediately make a writable clone of the data instead of scping the data around.