RoboCopy slower than Xcopy? (Windows 10)

Question

I'd been using xcopy in a backup script but every so often xcopy would fail with "insufficient memory" when a pathname would sneak into the backup set that exceeded 254 characters. Lots of advice on the web said xcopy was deprecated for robocopy and suggested using robocopy instead.

I switched to robocopy and it works fine but runs dramatically slower. These are big backup sets and the xcopy version ran for 6 hours, which was OK for overnight. But the robocopy version runs for 11 hours which means it's still running the next morning! Is there a way to speed up robocopy OR is there a way to force xcopy to ignore long file names and keep going? Here's sample code, old and new:

xcopy S:\SharedFiles E:\NAS\SharedFiles /c /f /i /s /e /k /r /h /y /d /j 1>> C:\utilities\alloutput.txt 2>&1

robocopy S:\SharedFiles E:\NAS\SharedFiles /e /j /np /fp /r:1 /w:1 1>> C:\utilities\alloutput.txt 2>&1

N.B. that I'm using the /C option in xcopy but that doesn't seem to stop xcopy from ending as soon as it encounters the long pathname.

EDIT I updated my script to do /r:1 /w:1. In my test run there were 78 errors requiring retries. This made a slight improvement but it's still way slower than the xcopy version. I've also tried it with and without /J, with no discernible improvement. I have not tried setting a threading limit, but AFAIK xcopy is single-threaded anyway and robocopy defaults to 8.

Answer 1

Usually, Robocopy will be the faster alternative in most cases however the most significant difference is that robocopy has a retry option where as xcopy does not retry on errors.

To verify this use the switch r:0 instead of r:1 (and w:0 to be safe but that shouldn't make any difference).

Other than that I know of the following performance switches:

     /COMPRESS : Request SMB network compression during file transfer, if applicable.
            /J : Copy using unbuffered I/O (recommended for large files).
    /NOOFFLOAD : Copy files without using the Windows Copy Offload mechanism.
        /IPG:n : Inter-Packet Gap (ms), to free bandwidth on slow lines.
       /MT[:n] : Multithreaded copying, n = no. of threads to use (1-128)
                 default = 8 threads, not compatible with /IPG and /EFSRAW
                 Redirecting output using /LOG is recommended for even better performance.

Depending on your specific scenario it's helpful to specifically turn them off/on (in your case - does using /j actually make a difference?).

Answer 2

After thinking about my previous answer, in which I experienced different results in output and execution times when adding the /MT flag, I started thinking that maybe your robocopy OUTPUT might be the culprit here.

I tested these 2 robocopy commands:

robocopy vbs vbs2 /e /j /np /fp /r:1 /w:1 1>> alloutput2.txt 2>&1

robocopy vbs vbs2 /e /j /np /fp /NC /NS /MT:16 /r:1 /w:1 1>> alloutput.txt 2>&1

The execution time for the first command was ~9 seconds, for copying 12k files of total size 660MB. The execution time for the second command was ~2.7 seconds.

The output is a bit different as well. The first command will output this:

New File            3056    C:\git\vbs\.editorconfig

The second will only output the filename:

        C:\git\vbs\.editorconfig

I suggest you try it, and increase the MT to 16, as your CPU could use it.