3

I recently built a system with a btrfs root partition. The most compelling reason for adopting btrfs compared to ext4, in my case, was live copy-on-write snapshots, with near-zero latency. Compared to ext4, when a full system backup entailed taking down the system, mounting from a live distribution, and building a partclone image on removable media, the promise of snapshots is that the snapshot can be captured on the backup media while the system is live.

To my astonishment, no tool exists, however, that will capture the entire snapshot to a single file on external media, such that if the system is restored from that file, all applications will have the same view of the filesystem as before the crash (except that the other snapshots or subvolumes are missing).

The documentation suggests mirroring the directory tree using a tool such as rsync, or using btrfs-send/btrfs-receive to capture the incremental changes on another system. In the first case, I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured. The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another btrfs file system is available, which is not always the case.

Are any suggestions available for saving or restoring a volume-level image, similar to a partclone file, but representing only a select subvolume?

Improve this question
1
  • The problem here is that identifying what is a subvolume relies on metadata that you by definition don't want to copy (you only want a subvolume), so I don't think such a tool would be easy to write: It would have to modify this metadata, and keep track while doing so.
    – dirkt
    Commented Jan 20, 2019 at 6:54

2 Answers 2

Reset to default
1

btrfs send and btrfs receive are exactly the tools you need.

I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured.

Have you tried the tools? I have had no such problems with btrfs. btrfs send and btrfs receive do not work with files, they work with entire subvolumes, having access to all Btrfs metadata natively.

The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another Btrfs file system is available, which is not always the case.

You will need another Btrfs filesystem only if you want to replicate the subvolume there. This is often desirable, so you can mount it and get access to its files. And this is unavoidable, if you want to backup your subvolumes incrementally. You need the exact previous snapshot(s) in both places to incrementally send the next one.

With full dump, however, you get a standalone stream, which is exactly an image (but not a browsable backup, unless you btrfs restore it). Work with it like with any other stream:

btrfs send /source/subvolume >/another/filesystem/subvolume-image   # just a file
# (or you can gzip it and/or send with nc on the fly, whatever)
# then later
</another/filesystem/subvolume-image btrfs receive /some/btrfs/directory

Where /some/btrfs/directory may belong to the same Btrfs filesystem as /source.

Improve this answer
4
  • To be clear, my requirements are 1) capture an entire, single subvolume, 2) into a single file (or small group), 3) that can be stored as regular files on any external FS of any type. The purpose is crash recovery after equipment failure (akin to partclone for full partitions), not live, browsable, replicated instances. I have not tried the send command because I understood that it is only useful for exchange between two live btrfs partitions. Are you saying that I can use the command as in your example to capture a minimal but full dump of all data and metadata for a single subvolume?
    – brainchild
    Commented Jan 20, 2019 at 7:57
  • @epl Are you saying that I can use the command as in your example to capture a minimal but full dump of all data and metadata for a single subvolume? – I believe so. Yes.
    – Kamil Maciorowski
    Commented Jan 20, 2019 at 9:31
  • That is very nice. The tutorials indicate that btrfs has no solution for backups, and other tools such as rsync are recommended. I wish that they instead would indicate that btrfs has a great backup system. To create a backup, one first simply makes a snapshot, and then dumps the snapshot to a file using btrfs send. To test this approach online, I suppose I could very easily dump a snapshot to a file and then it restore to an empty subvolume from that file?
    – brainchild
    Commented Jan 21, 2019 at 1:34
  • @epl btrfs send looks OK when it comes to a full copy of a single subvolume, but for many of them sharing data blocks, and for incremental backups, the situation gets complicated. See this answer. In such case storing backups as files leads to problems: (1) Restoring must be done in proper sequence. (2) Incremental backups require you to store new files, while you cannot really delete old ones. (3) Non-incremental backup transfers some data again. // Unless the file holds a Btrfs filesystem you mount and btrfs restore to on the fly.
    – Kamil Maciorowski
    Commented Jan 21, 2019 at 5:55
0

I'm surprised the original question has gone unanswered for this long.

If what you need is an external subvolume backup as a file so that it can be stored in any FS you can treat the output of btrfs send and input of btrfs receive as regular streams and redirect them to/from a normal file with dd.

I imagine something along the following lines would answer your question:

For backing up to file:

# create read-only temporary snapshot of volume
btrfs subvolume snapshot -r /path-to-sv/subvol /path-to-sv/subvol.ro

# send the subvolume to file, compress with parallel ZSTD & monitor progress
btrfs send /path-to-sv/subvol.snap.ro | \
    pv -c | pzstd -16 |  pv -c | \
    dd of=/path-to-external-backup/subvol.zstd.back

# delete read-only snapshot
btrfs subvolume delete /path-to-sv/subvol.ro

To restore subvolume from backup we run the process in reverse:

# read backup file and decompress the stream, redirect to temporary read-only snapshot
dd if=/path-to-external-backup/subvol.zstd.back | \
    pv -c | pzstd -d | pv -c | \
    btrfs receive /path-to-sv/

# make a RW subvolume 
btrfs subvolume snapshot subvol.ro subvol

# delete temporary snapshot
btrfs subvolume delete /path-to-sv/subvol.ro

Notice btrfs receive only takes the path to the destination folder as argument, since the name of the subvolume is passed in the stream with the rest of the data. Of course monitoring with pv -c is entirely optional, as is using (de)compression.

I tried it on my linux box and it seemed to work without issue.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .