1

Hi since I have searched stack overflow and did not get info about this, I need to ask again with specific requirements. I have an old storage with approx. 4TB of data and I would like to sync it with new storage based on ZFS (TrueNAS core). This data on old storage gets changed often, new gets created and some of them gets deleted. since I cannot use ZFS send/receive from the old storage which would be a best option as I would do a block by block copy plus snapshots, I need to see a possibilities of an rsync.

Now rsync has several flags and most of examples that I found out are using

rsync -avr /source /destination

This would be a good thing to do if I would make on time archive and perhaps archive those new files which are created, but since old data is sometimes deleted and some current files which are located are being updated by programs (meaning content and timestamp is changed), I need to see an option which would include archive of newly created files, delete old files on new ZFS storage which is also deleted on the old "source" storage and also update of the modified files from source to destination storage..

Reading the manual rsync command should look like this:

rsync -avu --delete /source /destination --progress

but since this is live data, I need to be 100% certain, and I cannot simply test and "try" this out...

Is there anyone who is really good when it comes to rsync to check and perhaps give me an advice how this command should look like, or perhaps make suggestion...

P.S. This rsync should be se daily in a cronjob so, with all append, delete and update options included...

Thank you in advance.

Improve this question

1 Answer 1

Reset to default
0

My best practice with rsync. I have been using this for several years with RHEL6, RHEL7 and Ubuntu to synchronize directories of file servers and payment transaction systems.

Notes: In the destination directory, everything is deleted that is no longer present in the source directory. No changes are made in the source directory. The trailing slash at the source directory is important. I assume that locally and remotely the user root is used.

With two local file systems:

rsync -S -vrltH -pgo --stats -D --numeric-ids --delete --partial /path/to/source/dir/ /path/to/destination/dir

With a local source and remote destination (via ssh):

rsync -S -vrltH -pgo --stats -D --numeric-ids --delete --partial /path/to/source/dir/ root@server:/path/to/destination/dir

For details see: man rsync

These options are derived from a script framework for creating backups called dirvish.

Improve this answer
5
  • Hi. Yes, it is a remote connection to another server. So I guess this bottom script will allow new created files on source to be synced, the modified files from the source to replace/update same files with old timestamp on destination, and delete all files on destination which are no longer on the source..., right? And this needs to be set into cron job to be executed daily...
    – dovla110010101
    Commented Nov 14, 2020 at 9:41
  • The way you described it, you understood me correctly.
    – Cyrus
    Commented Nov 14, 2020 at 9:54
  • I have just one question. Which flag is for update, meaning that it will update those files which were modified. Partial flag only continues to resume the transfer if I understood correclty, and delete part is OK, but update of the files with newer timestamp on source in comparison to the destination, which flag does that... Also which flag does copy new files?
    – dovla110010101
    Commented Nov 16, 2020 at 11:32
  • 1
    The behavior to update a file on the destination system if it is different from the source and copy a file if it does not yet exist on the destination system is rsync's default. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences in the data. (source: man rsync)
    – Cyrus
    Commented Nov 16, 2020 at 18:29
  • Hi again, than perhaps I miss interpret rsync manual, I always thought that -u flag is the flag (or --update) used to update files with modified timestamp and -a for archive is doing the "append" of the new files... Well then, I have learned something new today. Thank you again for clarification and also for correcting me.
    – dovla110010101
    Commented Nov 17, 2020 at 19:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .