How can I check a file has been copied fully to a folder before moving it using python

Question

I'm currently working on a project that adds images to a folder. As they're added they also need to be moved (in groups of four) to a secondary folder overwriting the images that are already in there (if any). I have it sort of working using watchdog.py to monitor the first folder. When the 'on_created' event fires I take the file path of the newly added image and copy it to the second folder using shutil.copy(), incrementing a counter and using the counter value to rename the image as it copies (so it becomes folder/1.jpg). When the counter reaches 4 it resets to 0 and the most recent 4 images are displayed on a web page. All these folders are in the local filesystem on the same drive.

My problem is that sometimes it seems the event fires before the image is fully saved in the first folder (the images are around 1Mb but vary slightly so I can't check file size) which results in a partial or corrupted image being copied to the second folder. At worst it throws an IOError saying the file isn't even there.

Any suggestions. I'm using OSX 10.11, Python 2.7. The images are all Jpegs.

Answer 1

I see multiple solutions :

1. When you first create your images in the first folder, add a suffix to their name, for instance, filexxx.jpg.part and when they are fully written just rename them, removing the .part. Then in your watchdog, be sure not to work on files ending with .part
2. In your watchdog, test the image file, like try to load the file with an image library, and catch the exceptions.