Copy interrupted and difficulties to take it over by relaunch with cp -u or rsync

Question

I have a huge data to transfer between 2 macOS (1.3 TB).

Unfortunately, my copy from mac1 to mac2 (by mounting the concerned directory of mac1 on mac2 as network directory), the copy with 'rsync' (used with parallel command) has been interrupted (855 GB has been copied on 1.3 TB for all the content).

Since this incident, I can't take over correctly the missing rest of what has not been copied.

I tried (on macOS 10.15.4 Catalina) :

parallel -j32 rsync -az {} /Users/home_henry_from_astro/ ::: /Volumes/henry/*

and

parallel -j32 /usr/local/opt/coreutils/libexec/gnubin/cp -urp {} /Users/home_henry_from_astro/ ::: /Volumes/henry/*

In both case, the copy seems to not be achieved, the command with cp -u doesn't say anything, same thing for rsync -az, except some errors of permissions which are not critical, at least I think.

I check regulary the size of the directory /home_henry_from_astro/ but size doesn't increase, always the same size : 855 GB

What does it happen ? How could I circumvent this issue which seems to be a hanging in the copy process ?

EDIT : I provide below more informations to debug this hanging :

Here the results of a : $ ps aux | grep cp

sh-3.2# ps aux | grep cp
root             39911  26.8  3.9  7914992 2603432 s004  R    10:05PM 264:46.62 /usr/local/opt/coreutils/libexec/gnubin/cp -urp /Volumes/henry/Phd_2019_2020 /Users/home_henry_from_astro/
root             39915  24.1  4.0  7857356 2661576 s004  U    10:05PM 280:13.09 /usr/local/opt/coreutils/libexec/gnubin/cp -urp /Volumes/henry/Projets_info /Users/home_henry_from_astro/
root             39848   0.0  0.0  4605780  15244 s004  S+   10:05PM   0:15.34 perl /usr/local/bin/parallel -j32 /usr/local/opt/coreutils/libexec/gnubin/cp -urp {} /Users/home_henry_from_astro/ ::: /Volumes/henry/COPIE_Stage_Incomplete_lors_de_la_copie_de_kawito_vers_astro ...

It is really weird since I have a running flag 'R' on the first process but I don't know other codes like 'S+' or 'U' for the other processes

What do they mean ?

Any help is welcome.

Answer 1

Ideally you'll enable the ssh service on the "remote" Mac and run rsync in proper client/server fashion. This is far far more efficient than mounting a remote filesystem.

Nevertheless, let's assume you're still mounting the remote filesystem. Simplify the command and see what's happening here. I've removed -z because it's ignored when copying between directory points that appear to be in the local filesystem, and added -v to see what's going on. You might need to add another -v to get additional output

rsync -av /Volumes/henry/* /Users/home_henry_from_astro/

You've copied 800+ GB of data already. The rsync process has to skip all of that. Now because you've used the -a (archive) flag it can skip files based on size and time. But it has to retransfer in entirety any file that has either a different size or timestamp. Not just the parts that are changed, but the entire file. If this is a large file it's going to take quite a while to copy. (This is where you can gain if you can run one rsync on each end of a network connection, as then only changed blocks need to be retransferred.) If you're watching, you can use the --progress flag to see what's going on, but don't use this flag if you're writing to a log file

rsync -av --progress /Volumes/henry/* /Users/home_henry_from_astro/