Linux queue of files

Question

I have a few files waiting to be processed by a daily cron job:

file1
file2
file3

I want the job to take the first file and then rename the rest. file1 should be deleted. file2 should be renamed to file1, and file3 should be renamed to file2.

I'm looking for a solution that would work with any number of files.

Is there a simple way to do this with a script? Or, taking a step back, is there a standard Linux technique for handling a queue of files?

Answer 1

It looks like you are trying to implement a simple queueing mechanism for processing work on an arbitrary number of files, treating the filenames as queue positions (so that file1 is "head"). I think you're taking the queue metaphor a bit too literally into the filesystem space, however, as doing renames for all those files is extremely expensive in terms of filesystem operations and race-condition prone to boot (what if more files are added to the queue as you are renaming the previous ones?). What you should do instead is simply track the filenames to be operated on in a side file (e.g. don't traverse the filesystem looking for work, but traverse your "queue file") and lock that file whenever you're removing or adding an entry. A nice side-effect of that approach is that your filenames can then have any names you like, they don't have to be "file1, file2, ..."

Answer 2

You can use simple bash script as follows. It first list the files in the "folder1" to data.txt according to time stamp it created. Then first file will be removed. At last second file will be renamed first file continuously.

#!/bin/bash

# List the files in Folder1 folder
ls -tr folder1/ > data.txt

# Removing the first file
rm -rf "folder1/`head -n 1 data.txt`"

#Renaming the Old file name to new file        
while IFS= read -r file 
        do
        if [ -f "folder1/$file" ];      then
                mv "folder1/$file" "folder1/$newFile"
        fi
        newFile="$file"

done