14

Windows Copying is a real mess ever since Windows Vista. Even Microsoft claims they've improved the performance, from a user perspective, it's not quite visible. Even with single file the copying window appears too much time for 'Calculating' and then finishing the copy(Even after 100% completion some times the dialog remains active).

At the same time, I was backing up some files in Ubuntu Linux. I felt it's really fast. Might be a feeling caused by faster UI updates.

I read an informative post from Jeff Atwood few years back on Windows File Copying. but what my specific questions are

  • Is (Ubuntu) Linux file performance is better than Windows-7?
  • Are both algorithms, Windows and Linux is making use of multiple threads and pipelining mechanism to improve the speed? If yes, which one is better?
5
  • 4
    In Windows use xcopy, or even robocopy. Much better. For example, with the latter you can make two passes, first to create all the directories but not copy any files, and second to copy the actual files, thus causing no fragmentation on the target.
    – Kerrek SB
    Commented Jun 27, 2011 at 10:19
  • 2
    Or RichCopy (blogs.technet.com/b/ken and download)
    – sehe
    Commented Jun 27, 2011 at 13:04
  • 1
    Depends also on the filesystem - your system presumably using something like ext3/4, which is competing with NTFS here. You might also try reiserfs and xfs - both of which are optimized for fast operations with small files.
    – new123456
    Commented Jun 27, 2011 at 15:29
  • 1
    @KerrekSB, please make your comment an answer so I can upvote it. (I did just +1 the comment, but you know what I mean.) That's the answer.
    – CarlF
    Commented Jun 27, 2011 at 16:52
  • 1
    It should be noted (albeit 8 years later!) that Robocopy /CREATE, although very useful to replicate a complete directory structure with empty files, does not by itself prevent fragmentation of files on the target. The structure is created in the MFT, but then the files' contents can be recorded anywhere. Normally even in one pass if Robocopy is running mono-threaded and copies one file at a time, there should be no fragmentation anyway (and running it multi-threaded is a bad idea as it severely reduces the performance on a HDD to HDD copy, since the mechanical drive has to “grind” a lot more).
    – GabrielB
    Commented May 26, 2019 at 2:15

4 Answers 4

7

The standard copy builtin in Windows isn't very good, but Windows comes with two advanced commands that are far more efficient: xcopy and robocopy. Especially when you need full control over advanced filesystem features like ACLs or alternate data streams, robocopy is the tool of choice.

As an example, if you are copying a large directory structure, you may wish for the target to be unfragmented. The standard invocation of robocopy is already quite good at that, but since directory entries are updated as subdirectories get created, you can still get fragmented directories. To prevent that, you can run robocopy in two passes, the first of which only creates the target directory structure without copying any file content, and the second to actually copy the file bodies.

Altogether, robocopy is very efficient and fully aware of NTFS features.

6
  • Is NTFS better efficient than ext3/4?
    – sarat
    Commented Jun 28, 2011 at 2:41
  • How a two pass robocopy help us to reduce fragmentation, as the real content will be copied later. The disk allocation is entirely controlled by file manager no?
    – sarat
    Commented Jun 28, 2011 at 2:45
  • I am doing some tricks like using Beyond Compare to copy files. I.e I can run multiple copying operations to maximize the bandwidth usage. But I am not sure how fragmented it will be
    – sarat
    Commented Jun 28, 2011 at 2:46
  • 2
    @Sarath: Regarding fragmentation: The directory entries grow as new files and subdirs are added to the directory. If you just naively create the directory and start putting files in, eventually the original directory record will be too small and has to be extended, but now there's no space because you already have the file data following it. With the /CREATE option, empty files and subdirs are created, which occupy no space and thus allow the directory entries to be written out in full contiguously.
    – Kerrek SB
    Commented Jun 28, 2011 at 6:22
  • 1
    @Sarath: As for comparing NTFS to Ext, I can't say, because NTFS is not open and the question of efficiency is rather vague. NTFS is certainly a fine piece of technology with a large number of features. There isn't really a choice, though, because you'll use NTFS in Windows and Ext (or BTRFS) in Linux and that's it.
    – Kerrek SB
    Commented Jun 28, 2011 at 6:29
6

Some of the speed you see from Ubuntu Linux is probably file caching. As far as I know, file copy in Linux makes no effort to minimize the use of file cache.

With Linux when you copy a file that is smaller than about 50% of your RAM, the entire file is copied into file cache and then written to disk as time permits. This appears to be very fast, but it forces other more useful files out of cache. These might be web cache files, emails, configuration files, font files, etc.

Windows XP used to copy without regard for file cache also. But in Windows Vista, Microsoft changed Explorer so that when it copies it uses a limited amount of file cache. This reduces the impact of large file copies on the rest of the system. But it also reduces the speed of even relatively small file copies to the speed of the hard disk.

2
  • 1
    Yea Ubuntu using the file caching mechanism. I also noticed the browsing is slightly slow while copying was in progress. I also noticed that the speed is consistently improved especially while copying large number of files
    – sarat
    Commented Jun 28, 2011 at 2:43
  • 1
    I think this is the answer that addresses the question almost completely. Commented Sep 9, 2017 at 7:10
2

In Windows, use Teracopy - it will solve the overall stupidity problem that Explorer has.

In Ubuntu, I found file copying to be very fast.

Still the "File Opertion" dialog does not go away when you are using removable drives. If you try to safely eject the drive immediately after the dialog disappears, you will get some error message. Data gets through though.

Update: Windows file operations are slow probably of all the integration with services and registry. Nautilus (in Gnome) is just a simple process I guess or at least does not have the load/interruptions that Explorer has.

4
  • Solve the stupidity by adding another GUI that must stay open :) In ubuntu, my experience is that you can dismiss the copy/move dialog and it will go to the system tray, showing a notification when done. Not sure about how Natty with Unity desktop does that
    – sehe
    Commented Jun 27, 2011 at 13:00
  • Teracopy gets itself loaded with Explorer. You do not have to keep it open. So, Teracopy is loaded on to memory without any user effort and it works unobtrusively. Ubuntu does not need anything like that because it has all that intelligence built into Nautilus i.e., "merge" and "skip all" options.
    – BZ1
    Commented Jun 28, 2011 at 3:28
  • +1: thanks for bringing TeraCopy to my attention then!
    – sehe
    Commented Jun 28, 2011 at 6:29
  • I was using a free and open source tool called fastcopy. [ipmsg.org/tools/fastcopy.html.en]
    – sarat
    Commented Jun 28, 2011 at 13:21
-1

Moving and copying files in Linux is naturally a quick operation. The reason you can move and copy files so quickly is because both source and destination are in the same partition. The operation only needs to change the inodes. Moving files between different partitions is still very quick (only a minute or so for a 1-gig file). In Windows, it takes some to move files to destinations on the same partition because file management in Windows is messy. (Think about file fragmentation and the way Windows formats hard discs.)

2
  • I agree that moving is pretty fast within the partitions. But copying always wanted to have a duplicated date in the disk and create new inodes. Also I doubt moving files between partitions is costly because there's physical file movement happening. Correct me if I am wrong.
    – sarat
    Commented Jan 19, 2012 at 6:06
  • @sarat Copying files between partitions (usually) does indeed involve deleting and writing them to a different place in the disk.
    – jjmontes
    Commented Jun 16, 2021 at 12:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .