I want to copy all files in a directory except some files in a specific sub-directory.
I have noticed that cp
command didn't have the --exclude
option.
So, how can I achieve this?
20 Answers
rsync
is fast and easy:
rsync -av --progress sourcefolder /destinationfolder --exclude thefoldertoexclude
You can use --exclude
multiples times.
rsync -av --progress sourcefolder /destinationfolder --exclude thefoldertoexclude --exclude anotherfoldertoexclude
Note that the dir thefoldertoexclude
after --exclude
option is relative to the sourcefolder
, i.e., sourcefolder/thefoldertoexclude
.
Also you can add -n
for dry run to see what will be copied before performing real operation, and if everything is ok, remove -n
from command line.
-
5Agreed, you can't beat the simplicity and power of
--exclude
Commented Feb 14, 2013 at 17:51 -
47is the
thefoldertoexclude
relative to thesourcefolder
or the current working dir? thanks– BeebeeCommented Aug 18, 2013 at 12:20 -
55It's relative to the source folder. This will exclude the folder source/.git from being copied. rsync -r --exclude '.git' source target– orkodenCommented Nov 14, 2013 at 18:55
-
25Maybe I'm wrong but I think it's a good practice to add "switches" before "parameters". Also the man page of rsync reports --exclude usable as with the "=" syntax or without. So to standarize across operating systems, I'd use
rsync -av --progress --exclude="thefoldertoexclude" sourcefolder /destinationfolder
- anyway upvote for the rsync instead of the find, as you can easily use absolute paths for the source while in the find it's trickier as it uses the{}
in the dst. Commented Feb 11, 2014 at 17:30 -
8
--exclude /node_modules/
will exclude node_modules anywhere in the source tree. So powerful while copying a bunch of JavaScript/Node.js project folders. Commented Jul 22, 2021 at 9:45
Well, if exclusion of certain filename patterns had to be performed by every unix-ish file utility (like cp, mv, rm, tar, rsync, scp, ...), an immense duplication of effort would occur. Instead, such things can be done as part of globbing, i.e. by your shell.
bash
man 1 bash
, / extglob.
Example:
$ shopt -s extglob $ echo images/* images/004.bmp images/033.jpg images/1276338351183.jpg images/2252.png $ echo images/!(*.jpg) images/004.bmp images/2252.png
So you just put a pattern inside !()
, and it negates the match. The pattern can be arbitrarily complex, starting from enumeration of individual paths (as Vanwaril shows in another answer): !(filename1|path2|etc3)
, to regex-like things with stars and character classes. Refer to the manpage for details.
zsh
man 1 zshexpn
, / filename generation.
You can do setopt KSH_GLOB
and use bash-like patterns. Or,
% setopt EXTENDED_GLOB % echo images/* images/004.bmp images/033.jpg images/1276338351183.jpg images/2252.png % echo images/*~*.jpg images/004.bmp images/2252.png
So x~y
matches pattern x
, but excludes pattern y
. Once again, for full details refer to manpage.
fishnew!
The fish shell has a much prettier answer to this:
🐟 cp (string match -v '*.excluded.names' -- srcdir/*) destdir
Bonus pro-tip
Type cp *
, hit CtrlX* and just see what happens. it's not harmful I promise
-
1@MikhailGolubtsov perhaps that's because globbing is not recursive and works one level at a time. Edited out. P.S: it works in
zsh
though.– ulidtkoCommented Jul 11, 2015 at 2:16 -
8Nice pro-tip! This way you can remove single items easily. Thanks a lot!– taffitCommented May 1, 2018 at 19:42
-
2BTW, to turn off extended pattern matching features in Bash, run
setopt -u extglob
. Commented Dec 10, 2019 at 0:45 -
1"... an immense duplication of effort..." shouldn't it be just a one-liner: exclude paths from the list that match a regex? How the file manipulating utilities like
cp
don't support this most simple and straightforward use case out of the box is beyond me. Thanks for the tip though!– ayorgoCommented Sep 30, 2020 at 7:26 -
1@ayorgo well yes it "should" — but in C, a oneliner can't do much: multiply some ints and maybe move a pointer, that's it. Even ignoring the source level, regex matching in C involves additional library dependency and additional machine code output — now multiply this by the number of commands, and you've got nontrivial (unbounded?..) overhead. At least that's my understanding why it was "refactored" to the shell; I can totally relate to the subpar UI aspect of it, but hopefully you can also see the technical justification now. Best wishes!– ulidtkoCommented Sep 30, 2020 at 13:09
Why use rsync
when you can do:
find . -type f -not -iname '*/not-from-here/*' -exec cp '{}' '/dest/{}' ';'
This assumes the target directory structure being the same as the source's.
-
13I think you need the
-path
argument to test path hierarchies, not -iname Commented Jun 5, 2012 at 0:34 -
6And you'll also need a semi-colon at the end:
find . -type f -not -path '*/not-from-here/*' -exec cp '{}' '/dest/{}' \;
Commented Feb 14, 2013 at 17:13 -
1Wow, it won't let me: "Edits must be at least 6 characters" ! Commented Feb 14, 2013 at 17:54
-
1@MatthewWilcoxson Meh. Those restrictions will be lifted, as soon as you gain a little more rep. I edited the answer accordingly. Thanks again! Commented Feb 14, 2013 at 19:24
-
3@Henning why not
rsync
? Coz it may be not present in the system! whilefind
,cp
is always on their places. Or you from kind of guys, who installed 2gigs of stuff to do simple things?– ReishinCommented Jul 6, 2018 at 14:20
cp -r `ls -A | grep -v "c"` $HOME/
-
1
-
Made a shell function which simplifies the usage for custom source path and exclusion of just one file or directory:
# $1 = source path # $2 = destination path # $3 = filter copy_from_source_to_destination_except_filter() { cp -r $(ls -A $1 | grep -v -w $3 | awk -v path=$1 '{printf "%s/%s ", path, $1}') $2 }
Commented May 5, 2019 at 16:28 -
3
-
1@Sérgio I haven't tested it but
cp -r "$(ls -A | grep -v "c")" $HOME/
should work. The command in the answer fails because therecp
operates on the output ofls -A | grep -v "c"
, which is unquoted and therefore breaks on spaces."$(…)"
is the same as"`…`"
but easier on the eyes. Commented Oct 23, 2020 at 17:10
The easiest way I found, where you can copy all the files excluding files and folders just by adding their names in the parentheses:
shopt -s extglob
cp -r !(Filename1 | FoldernameX | Filename2) Dest/
-
13
-
10shopt -s extglob (execute this to enable ! in cp, rm and others)– imkostCommented Oct 12, 2014 at 6:54
-
1@geneorama This happens if history substitution is enabled. serverfault.com/a/208414/352016– mbomb007Commented Oct 1, 2020 at 17:16
-
1
-
It should be
cp -r !(Filename1|FoldernameX|Filename2) Dest/
without the spaces inside the parantheses. Commented Mar 22 at 9:19
It's relative to the source directory.
This will exclude the directory source/.git
from being copied.
rsync -r --exclude '.git' source target
-
What is the difference/improvement compared to the top answer? Commented Jan 13, 2020 at 13:06
-
1
-
I feel like the '-a' in the first answer is better than plain old -r: explainshell.com/explain?cmd=rsync+-a Commented Jul 27, 2021 at 18:09
-
1@reducingactivity nothing much but simple to digest as the expression is shorter, just my personal preference– Goran B.Commented Sep 16, 2021 at 20:50
Expanding on mvds’s comment, this works for me
cd dotfiles
tar -c --exclude .git --exclude README . | tar -x -C ~/dotfiles2
-
The nice thing about tar, you can use exclude.tag files to ignore directories stackoverflow.com/a/13280610/722796 also gnu.org/software/tar/manual/html_node/exclude.html Commented May 9, 2021 at 0:25
rsync
is actually quite tricky. have to do multiple tests to make it work.
Let's say you want to copy /var/www/html
to /var/www/dev
but need to exclude /var/www/html/site/video/
directory maybe due to its size. The command would be:
rsync -av --exclude 'sites/video' /var/www/html/ /var/www/dev
Some caveat:
- The last slash
/
in the source is needed, otherwise it will also copy the source directory rather than its content and becomes/var/www/dev/html/xxxx
, which maybe is not what you want. The the
--exclude
path is relative to the source directly. Even if you put full absolute path, it will not work.-v
is for verbose,-a
is for archive mode which means you want recursion and want to preserve almost everything.
-
a simple solution that take cares of special characters and white spaces– SérgioCommented May 28, 2019 at 1:51
-
1Thanks for explaining parameters, unlike the current top answer! Commented Jan 13, 2020 at 13:07
-
-
1@ddzzbbwwmm You probably figured it out by now, but for posterity's sake: you can add multiple
--exclude
flags, like:--exclude 'foo' --exclude 'bar'
Commented Sep 2, 2021 at 21:48
cp -rv `ls -A | grep -vE "dirToExclude|targetDir"` targetDir
Edit: forgot to exclude the target path as well (otherwise it would recursively copy).
-
7
rsync
rsync -r --verbose --exclude 'exclude_pattern' ./* /to/where/
and first try it with -n option to see what is going to be copied
-
2What is the difference/improvement compared to the top answer? Commented Jan 13, 2020 at 13:06
I assume you're using bash or dash. Would this work?
shopt -s extglob # sets extended pattern matching options in the bash shell
cp $(ls -laR !(subdir/file1|file2|subdir2/file3)) destination
Doing an ls excluding the files you don't want, and using that as the first argument for cp
-
13You can skip the extra
ls
and simply docp !(file1|file1) dest
. Commented Jan 12, 2011 at 15:08 -
Do not use -laR. it add string that interfere with cp.
cp $(ls folder/!exclude_folder0|exclude_folder1)) dest
– LALCommented Dec 9, 2015 at 21:14
Just move it temporally into a hidden directory (and rename it after, if wanted).
mkdir .hiddendir
cp * .hiddendir -R
mv .hiddendir realdirname
-
1Not pretty maybe – but this is the only option I’ve found here which works with
cp
and a standard POSIX shell likesh
.– tomekwiCommented Sep 1, 2015 at 11:38 -
1This answer is dramatically underrated. This is the most compatible, easiest to read and easy to understand answer. Kudos, I don't know why I didn't think of it. Commented Jun 13, 2020 at 3:37
-
Thanks @RobertTalada, how far will the answer go from now? ️🌈 Commented Jun 13, 2020 at 8:02
-
1Obvious drawback is that you might be avoiding copying something because it's too big.– RobinoCommented Jun 28, 2021 at 7:26
rsync
went unavailable for us. Below is an alternative that works.
tar -cf - --exclude='./folder' --exclude='./file.tar' ./source_directory | tar -xf - -C ./destination_directory
Another simpler option is to install and use rsync which has an ``--exclude-dir` option, and can be used for both local and remote files.
This is a modification of Linus Kleen's answer. His answer didn't work for me because there would be a . added in front of the file path which cp doesn't like (the path would look like source/.destination/file).
This command worked for me:
find . -type f -not -path '*/exlude-path/*' -exec cp --parents '{}' '/destination/' \;
the --parents command preserves the directory structure.
ls -I "filename1" -I "filename2" | xargs cp -rf -t destdir
The first part ls
all the files but hidden specific files with flag -I
. The output of ls
is used as standard input for the second part. xargs
build and execute command cp -rf -t destdir
from standard input. the flag -r
means copy directories recursively, -f
means copy files forcibly which will overwrite the files in the destdir
, -t
specify the destination directory copy to.
10 years late. Credits to Linus Kleen.
I hate rsync
! ;) So why not use find
and cp
? And with this answer also mkdir
to create a non-existent folder structure.
cd /source_folder/ && find . -type d -not -path '*/not-from-here/*' -print -exec mkdir -p '/destination_folder/{}' \;
cd /source_folder/ && find . -type f -not -path '*/not-from-here/*' -print -exec cp -au '{}' '/destination_folder/{}' \;
It looks like cd
ìs necessary to concat relative paths with find
.
mkdir -p
will create all subfolders and will not complain when a folder already exists.
Housten we have the next problem. What happens when someone creates a new folder with a new file in the middle of it? Exactly: it will fail for these new files. (Solution: just run it again! :)) The solution to put everything into one find
command seems difficult.
For clean-up: https://unix.stackexchange.com/q/627218/239596
-
Gave it a try. Current version of your answer creates in the first step directories. Because this step currently contains
-not -path '*/not-from-here/*'
, it will create directory./not-from-here
. Probably, this is not intended. Therefore, for the first step (directory creation), you probably want-not -path '*/log'
instead.– AbdullCommented Apr 13, 2021 at 18:05
I use a "do while" loop to read the output of the find command. In this example, I am matching (rather than excluding) certain patterns since there are a more limited number of pattern matches that I want than that I don't want. You could reverse the logic with a -not
in front of the -iname
flags:
find . -type f -iname "*.flac" -o -print0 -iname "*.mp3" -print0 -o -iname "*.wav" -print0 -o -iname "*.aac" -print0 -o -iname "*.wma" -print0 | while read -d $'\0' file; do cp -ruv "$file" "/media/wd/network_sync/music/$file"; done
I use the above to copy all music type files that are newer on my server than the files on a Western Digital TV Live Hub that I have mounted at /media/wd
. I use the above because I have a lot of DVD files, mpegs, etc. that I want to exclude AND because for some reason rsync looks like it is copying, but after I look at the wd device, the files are not there despite no errors during the rsync with this command:
rsync -av --progress --exclude=*.VOB --exclude=*.avi --exclude=*.mkv --exclude=*.ts --exclude=*.mpg --exclude=*.iso --exclude=*ar --exclude=*.vob --exclude=*.BUP --exclude=*.cdi --exclude=*.ISO --exclude=*.shn --exclude=*.MPG --exclude=*.AVI --exclude=*.DAT --exclude=*.img --exclude=*.nrg --exclude=*.cdr --exclude=*.bin --exclude=*.MOV --exclude=*.goutputs* --exclude=*.flv --exclude=*.mov --exclude=*.m2ts --exclude=*.cdg --exclude=*.IFO --exclude=*.asf --exclude=*.ite /media/2TB\ Data/data/music/* /media/wd/network_sync/music/
tar -c | tar -x
?