1

I have an old hard drive and basically clone it to a new hard drive using Grsync or Timeshift. Because the older hard drive is likelier to fail, I'd like to use the newer one as my in-use hard drive and the older one just as my backup drive where all changes are synced.

First asked about this at the Vorta GUI repo since ultimately I'd like to use only one GUI for all backup tasks. Because that tool doesn't (yet) have this functionality, I'd like to use another open source GUI. Further differences to related questions and to what some available tools I tried offer is described below:

  • Syncthing: For strange reasons it's not really supporting backups to local destinations (syncing local hard drives or folders); my second hard drive is local
  • Unison: I'd like the syncing to be always-on automated synchronization without any triggering needed
  • Rsync GUIs: These seem to have the same problem as Unison (recommended here): I'd like to have a directory monitor daemon / File System Monitor that detects changes on the large HDD I use as they are made and then at some point (immediately or specified intervals) syncs them to the second hard drive automatically. With tools like Grsync ("Delete on destination" option enabled) or BackInTime (unmaintained) this can only be done manually. I'd like the files to be exactly the same so I could mount and use the fully-encrypted drive exactly as if it was the old one if the new one fails (for example file permissions and everything else should stay the same). It should be efficient and for example not scan the whole drive for changes anew every time.

To me this seems like the most intuitive way desktop users could backup their full hard drives without versioning so I was surprised it's not very easily possible with most existing GUI tools. Even worse, other questions (related but not the same!) such as this seem to be only about command-line solutions for niche applications techies and even without a concrete rsync command one could use or information how to implement it. I use Debian with KDE.

Improve this question
16
  • I can only think of a somewhat involved scripting solution to this, given the requirement for it to be either immediate or at a later time (when the 2nd disk gets plugged in). The basic idea is that you stand up an inotifywatch on the primary's mountpoint and either trigger the sync of each file/directory as it receives a write_close event (note that a recursive watch won't pick up changes in newly created directories, so you'd have to restart the watch when a new dir gets created) or write the name to some tempfile and have it synced by a mount-script.
    – tink
    Commented Jan 1 at 19:53
  • @tink I'd prefer it only syncs at certain intervals because then the second disk can be in standby most of the time which may be good for its lifespan, the main reason is that I'd like it to be silent so if it syncs all the time it'll be too loud than if it was doing that only every few hours. There must be a tool for that - I thought this was the most reasonable way to backup for 90% of people if you either don't need versioning or want to have nonprimary HDD (such as an external HD) get backed up. The dirmon would simply log which files have been modified or changed and then let it back up.
    – mYnDstrEAm
    Commented Jan 1 at 20:49
  • 1
    I'm not convinced that it "being on stand-by" is the best thing to assure longevity. Most HDD failures I've encountered in my career occured when a machine got rebooted, AKA the spindle stopped and restarted. But yes, you could use inotify to just generate that list and then act upon it say every thirty minutes if that's what you desire.
    – tink
    Commented Jan 1 at 20:52
  • In addition to what tink said... You're now saying the opposite of what you had said in your post – you specifically excluded Unison and Rsync because they weren't continuous, and now you don't want continuous sync. If your goal is to only sync at certain intervals (which literally means being triggered by a scheduled timer), then what are the actual issues with using e.g. Unison?
    – grawity_u1686
    Commented Jan 1 at 20:57
  • 1
    Try to use FreeFileSync with a scheduled task, as described here: freefilesync.org/manual.php?topic=schedule-batch-jobs There could be issues though with system files which are access protected during execution, so I'd run a second batch job on shutdown for all those files.
    – 1NN
    Commented Jan 8 at 8:24

1 Answer 1

Reset to default
1

Here's the best I came up with so far and I'll edit it if I find a better way:

  • Option #1: Convert the disk to RAID1 - The three problems with that are that:

    • If you delete something on the source disk it will be deleted on the second as well but for desktop use-cases it's better if you can then restore the directory or file for some time until the next backup (this is useful for servers where this can avoid downtimes but not good for personal backups)
    • There are quite high chances you'll lose your data so this should only be done if you backed up your data on yet another third drive
    • There are some step-by-step tutorials for that (where it's difficult to see which is up-to-date and fits your requirements) but no GUI

  • Option #2: Using Grsync. This seems to be the only advanced GUI for rsync that is still developed since BackInTime and LuckyBackup are not actively developed anymore. I haven't checked if the Timeshift GUI can do what I'm trying to as well and there are probably some less developed GUIs for rsync that can do the same.

    1. First select source and target directory and add an extra slash / after the source directory once you did so. Make really sure these are configured correctly because files on the target directory (second line) will be deleted.

    2. Then these options are enabled for syncing the specified directory which is the whole drive (changes from default settings; please correct any that are wrong):

      • "Delete on destination"
      • "Preserve owner"
      • "Preserve group"
      • "Preserve timestamp"
      • "Preserve permissions"
      • "Copy symlinks as symlinks" in the Advanced tab, fourth from top on the left (if you use shortcuts for files this ensures they are also there in the the backup directories)
      • "Copy hard links as such" (not including these two options could lead to the backup being larger than the source dir and maybe run full earlier)
      • ("Verbose" is disabled so it doesn't produce large logs)
      • ("Only compare file-size" may make it faster but I haven't tested this; disabling "Show progress" may be useful too)

    3. One then simply has to manually regularly start the program with the saved profile and click on the play button to refresh the sync backup. Save the profile by going to Session->Export so you can use always the same backup profile.

    The problems with this solution are:

    • It can't do scheduled or automatic backups. I would submit a feature request about it (linked here) but it seems like it doesn't even have a modern git repo (the first thing is Grsync or a tool like it to get on GitHub or Gitlab).
    • If you also want your root permission files synced you still need to run a command after the sync run since there is no root GUI for Grsync (there is one for BackInTime): in Grsync go to File->R sync command-line, copy the command into a terminal but add sudo in front of it before running it (as the root user). Also remove --progress there if you have that enabled. For me the command is sudo rsync -r -t -p -o -g --delete -l -H -s dir1/ dir2 where dir1 for me is just the root directory of disk1. (Instead of using Grsync you could also just use the commandline until there is at least one GUI that allows for all functionality.)
    • I don't know how well suited it is for syncing large whole drives. Since this question may get closed this may remain unclear. I would imagine that a file watcher type of daemon that logs which files you have changed as you change them would make it much more suitable for that and I don't think rsync/Grsync has that functionality. The same applies to when you only move files around where it may delete them in the destination to copy them over another time rather than just moving the folder or files there.
Improve this answer
4
  • I feel you might mix up two separate concepts: resiliance against hardware failure and have backup history to undo something. Raid is against hardware failure and seems exactly what you want with your old drive. But you ALSO want pointwise backup to (at best: off site) media, which is something else! Good esp. when you need a history of past snapshots. Think a minute about how differnt a restore process for each would be: on failing hardware or for a need to restore past snapshot. You need both? Then separate things. For using raid1 I actually would try to convince you against gui ;-).
    – tverrbjelke
    Commented Jan 11 at 12:30
  • No, I'm not looking for a backup history and made that clear in the question. Synchronizing already means you don't have backup revisions. I explained why Raid currently can't be the solution: mainly no GUI and data loss risk (once these two issues are solved it could be the better answer here). I don't want a history of snapshots and can't have one due to disk space. Restore process: one drive fails, I use the other until I have one new drive. GUI is absolutely required and it's completely precluded if there is no such, and it will never be widely used by the general population without a GUI.
    – mYnDstrEAm
    Commented Jan 11 at 13:06
  • 1
    Your rsync suggestion could be simplified to sudo rsync --archive --delete dir1 dir2
    – Chris Davies
    Commented Feb 3 at 17:59
  • That's useful; it only differs from the above command by the -D flag (--archive equates to -rlptgoD). I think that it also "transfer character and block device files" makes it less suitable for when this is used for syncing the root partition but shouldn't be a problem in this case. There are some further parameters for rsync like especially -H and -N that could be useful but seem to be missing in Grsync. I still have the problem that the target device doesn't free disk space after deleting files.
    – mYnDstrEAm
    Commented Apr 7 at 22:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .