I have a script that has the following steps:
- 1 mirror a remote server with lftp
open ftps://'[name]':'[pwd]'@[remote_host]
set ssl:check-hostname no
mirror --delete-first --only-newer /ExchangeSets
/home/sandbox_autopilot/Exchangesets
exit
- 2 next I sort the files based on the start of the filename using the find command, and copying them to a folder.
find /home/sandbox_autopilot/Exchangesets -iname '1R4*.000' -exec cp -u --target-directory /home/sandbox_autopilot/1R4/ {} \;
find /home/sandbox_autopilot/Exchangesets -iname '1R5*.000' -exec cp -u --target-directory /home/sandbox_autopilot/1R5/ {} \;
find /home/sandbox_autopilot/Exchangesets -iname '1R6*.000' -exec cp -u --target-directory /home/sandbox_autopilot/1R6/ {} \;
- 3 I do i whole bunch of GIS alterations to the file in the folders: 1R4, 1R5, 1R6 that are not relevant to my question.
significant things:
- after having mirrored the remote server the folder /home/sandbox_autopilot/Exchangesets is ordered with subfolder names that start with 4 digits, folders containing newer files have a higher 4 digit (see example below) start than older folders.
- There are multiple versions of the same file in the folder structure in /home/sandbox_autopilot/Exchangesets. Requested behaviour from the "find -exec cp" command is that the newest version of the available file is put in the target directory.
Example of multiple files in the folder /home/sandbox_autopilot/Exchangesets structure:
find . -name 1R65Y842.000
./5704b_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
./5721a_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
./5673b_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
./5618_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
./5802b_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
./5646a_[projectname]/ENC_ROOT/1R/6/1R65Y842/1R65Y842.000
note: the [projectname] are all different but blanked-out in this example because of privacy.
The problem:
The "find -exec cp" command does not give me the newest file in the 1R6 folder.
I think this is what happening.
- In the original folder structure the date on the file is correctly passed by the "lftp mirror" command. So the newest file has the latest date.
- When the "find -exec cp" command finds a file and copies it to the relevant 1R folder. the filedate is set to now(). Then when the "find -exec cp" command finds a file with the same name that is newer it won't copy, because the date in de target directory is newer (now()) then the filedate of the file that needs to overwrite the file in the target directory, thus rendering the -u function on cp useless.
The solutions that I am thinking of:
- can I prevent the cp command of changing the date of the file when it copies it to the target directory? So the cp -u can evaluate the correct date and put the latest version of the file in the target directory?
- Would it help to symlink instead of making a actual copy
- Is there an option on find that it evaluates the found versions of the file, and only executes the copy command on the latest version of the file?
My humble request:
Can anybody help me in the right direction?
