I have a large directory tree and want to copy only files with specific names. I also want to preserve the directory structure, incrementally update, and maintain metadata, so rsync
seems like the natural choice. However, I can't get the syntax to work correctly. Consider the following example structure:
$ find .
.
./filter.txt
./dest
./source
./source/no
./source/fold1
./source/fold1/yes
./source/fold1/no
./source/fold2
./source/fold2/no
./source/fold2/yes
So I have a source folder with subfolders and files named yes
(copy) and no
(do not copy). Based on some other questions (eg. https://serverfault.com/questions/770728/rsync-exclude-all-directories-except-a-few) I have been trying to use a filter file that includes files I want but excludes everything else. I have tried several things, but a simple filter file that is not working is:
$ cat filter.txt
+ /fold1/yes
- *
$ rsync -avv --progress --include-from=filter.txt source/ dest/
sending incremental file list
[sender] hiding file no because of pattern *
[sender] hiding directory fold1 because of pattern *
[sender] hiding directory fold2 because of pattern *
delta-transmission disabled for local transfer or --whole-file
total: matches=0 hash_hits=0 false_alarms=0 data=0
sent 59 bytes received 86 bytes 290.00 bytes/sec
total size is 0 speedup is 0.00
Changing the order or using slightly different syntax (eg. */yes
, yes
) do not change anything. In the final implementation, I would like to be able to include all yes
files without describing their folders, and there are many other files besides no
so excluding them individually is not a good solution. It seems like the - *
filter line is excluding everything regardless of whatever has been included above or below it, but that is inconsistent with the other information I've seen.
How can I rsync only specific-named files (in subfolders) while excluding everything else?
I have seen some similar questions but (as discussed above) they do not seem to solve my problem, eg.: https://serverfault.com/questions/770728/rsync-exclude-all-directories-except-a-few https://serverfault.com/questions/1063730/rsync-exclude-all-files-in-dir-except-specific-files.
Edit 1 Per @harrymc, a filter that explicitly includes the folder and file works for that file:
$ cat filter2.txt
+ /fold1/
+ /fold1/yes
- *
$ rsync -avv --progress --include-from=filter2.txt source/ dest/
sending incremental file list
[sender] hiding file no because of pattern *
[sender] showing directory fold1 because of pattern /fold1/
[sender] hiding directory fold2 because of pattern *
[sender] showing file fold1/yes because of pattern /fold1/yes
[sender] hiding file fold1/no because of pattern *
delta-transmission disabled for local transfer or --whole-file
[generator] risking file fold1/yes because of pattern /fold1/yes
./
fold1/
fold1/yes
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/3)
total: matches=0 hash_hits=0 false_alarms=0 data=0
sent 157 bytes received 178 bytes 670.00 bytes/sec
total size is 0 speedup is 0.00
Edit 2 A generic filter that satisfies my needs is, based on the accepted answer:
+ */
+ */yes
- *
This automatically includes every yes
file while excluding everything else.
filter.txt
as line 1 :+ /fold1/
, line2 :+ /fold1/yes
.yes
files without specifically listingfold1/yes
,fold2/yes
, and so on. In my real use case I have several types ofyes
files, manyno
files, and many folders.+ /fold1, +/fold1/yes
is different than simply+/fold1/yes
it might help me build my own solution.yes
?yes
would be/*/yes
. In the real example, it could be/*/yes
and/*/also
: many files all with the same name(s).