3

I found that Amazon's Web Services, especially Glacier, are a very cost-effective way of storing data online. I'm wondering whether I can use them to backup my data with Borg Backup, which is an incremental backup program.

Can I backup with borg to Amazon Glacier, and what is the easiest way to do it ?

Improve this question

2 Answers 2

Reset to default
3

You can't back up directly to Glacier with Borg.

You can back up locally, then use the AWS S3 commands to upload to S3, or an "S3 drive" type script. From there you can use S3 lifecycle rules to move the files to Glacier. Personally I use a script to upload the files to Dropbox, because I wanted my backups in a different location / company in case my AWS account for some reason becomes inaccessible.

The backup scheme you suggested could waste disk space if done the simplest way. Borg and Attic both remove backup files in order to meet your configured backup schedule. For example it can keep daily backups for a week, then weekly backups for six months, then monthly backups. So over time the backup files created are removed on the local file system. If you archive everything to Glacier you'll have a lot of files in there that aren't actually needed. It doesn't hurt, but it will cost you money.

I don't have a great solution. You could sync your files, but this means if a virus or an accident and sync them you could lose your backups. My inelegant solution is to manually delete extraneous files occasionally, but this wouldn't scale. Interested in suggestions to improve it.

Update: I've published my guide to backing up from AWS EC2 to Dropbox, which would work for backing up to other destinations as well.

Improve this answer
3
  • Seems like I'm not the only one struggling for a good backup strategy :D
    – Hey
    Commented Nov 28, 2016 at 16:45
  • I'm happy enough with my approach, but automatic pruning of unnecessary files would be nice. However doing it automatically increases the chance of losing backups through error, so maybe it is best either paying for the storage or doing it manually.
    – Tim
    Commented Nov 28, 2016 at 17:41
  • 1
    @YdobEmos I've updated my answer with a detailed guide to backing up to Dropbox from AWS with Attic. It should work almost identically with Borg.
    – Tim
    Commented Jan 14, 2017 at 19:17
0

I think the best way for me is to have a S3 bucket and use rclone to mount it as a file system.

  1. So you configure the S3 bucket in rclone. When configuring, you can pick the storage class you want.

Personal note: about Glacier, do check the costs of downloading data and of making requests, don't look solely to the cost of storing data.

  1. Then, you use rclone mount to mount that bucket somewhere in your filesystem. You can use this systemd service to auto-mount it everytime you reboot, it is very good and well explained.

With that, when you put data on your mounted folder, it will go automatically to the S3 bucket and won't occupy local space of your machine.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .