error: linux "find" finds/renames not all directories containing string at first run

Question

I run find on a file structure (several hundreds of folders, about 4 layers deep) with commands like: find "/media/$disk/_all" -depth -type d -exec/execdir rename -d 's/ 1000k/ [1000k]/' "{}" \;. I noted repeatedly that some occurrences of folders containing 1000k remain unrenamed, I noted some of them got renamed on second run of the command. Why not all on the first run?

Also, I noted often output has "Can't rename /media/some_original /media/some_renamed: No such file or directory", after that I check and some_original got successfully renamed to some_renamed.

First error in my guess occurs due to find not going through all tree, second is due to find going and invoking exec twice for some/all? entries. I added -depth option later to be sure of "child" folder being renamed before "parent", but it did not help (and in my case I have not noted several levels due to be renamed anyway when I noted the error).

I was not able to find answer via web search. What could be the issue here?

System is Linux Mint 20.1.

Added: I've checked I have not found any links in the structure, using ls -lR /path/to/folder | grep ^l from https://stackoverflow.com/questions/8513133/how-do-i-find-all-of-the-symlinks-in-a-directory-tree.

Disk is mounted like that: type ext4 (rw,nosuid,nodev,relatime,stripe=8191,uhelper=udisks2)

Answer 1

Preliminary notes

• This answer is an attempt to explain what may have happened, not a firm explanation what happened for sure. This is because:

  • the OP has tried different variants of the command (with or without -depth, with -exec or -execdir), the exact command posted in the body of the question should be taken with a grain of salt;
  • each try possibly modified the directory tree, but there is no log stating what renaming happen because of what exact command;
  • similarly there is no log of errors;
  • after analyzing the problem (in comments and chat) we noticed there are few different issues that generate similar error messages (and therefore it's easy to assume there is just one issue).

  For these reasons an attempt to create a single MCVE (in a form of one exact directory tree and one exact find command) failed. We agreed it will be the best to create an answer that covers possible issues, with future readers in mind. This answer is such answer.

• There is more than one rename. The syntax in question works with rename being a Perl script associated with Larry Wall and Robin Barker. I assume this is the rename used. The issues covered in this answer are not specific to this particular rename though.

---

What may have happened

There at few different issues that can generate No such file or directory when you use find with rename.

First, let's simplify the example. Let the directory structure be:

/a/b/c.1/c.2/d

and the command be:

find /a/b -exec rename s/c/X/ {} \;

This command will yield:

find: '/a/b/c.1': No such file or directory

This is because without -depth find processes c.1 before it tries to descend to c.2. When processing c.1, rename renames it to X.1, but find is not aware the name changes. It tries to work with c.1 in order to process deeper files but c.1 no longer exists. This is very similar to using find with rm -r.

The second iteration of the command will see /a/b/X.1/c.2/d and successfully rename c.2 to X.2. Nevertheless it will complain about c.2 not being there. The third iteration will do nothing and complain about nothing.

You need to tell find to process files in a directory before doing anything to the directory itself. This is what -depth is for.

---

Even with -depth there may still be problems. Again, having this structure:

/a/b/c.1/c.2/d

and running the "fixed" command:

find /a/b -depth -exec rename s/c/X/ {} \;

we will get:

Can't rename /a/b/c.1/c.2/d /a/b/X.1/c.2/d: No such file or directory
Can't rename /a/b/c.1/c.2 /a/b/X.1/c.2: No such file or directory

It's not immediately obvious the messages come from rename but they do. The problem is d is processed first and rename wants to change /a/b/c.1/c.2/d into /a/b/X.1/c.2/d. It's like mv /a/b/c.1/c.2/d /a/b/X.1/c.2/d, you cannot do this if /a/b/X.1/c.2/ does not exist. And later (while processing c.2) you cannot mv /a/b/c.1/c.2 /a/b/X.1/c.2 because X.1 does not exist (yet).

^{(You may be surprised rename by default tries something that cannot succeed. Note mv /a/b/c.1/c.2/d /a/b/X.1/c.2/d cannot succeed when X.1 doesn't exist. Still this command is sane when X.1/c.2/ already exists along c.1/c.2/. The default behavior of rename is equally sane in this case.)}

In our case X.1 appears when c.2 is silently renamed to X.1 but this happens later. The second iteration of the command will complain while processing d, it will manage to rename c.2. The third iteration will do nothing and complain about nothing.

Note if you had /c.0 instead of /a and the command was find /c.0/b … then no rename would succeed and any number of iterations would be futile.

The problem is rename s/c/X/ tries to change the first c in the path, even if this c is in a directory component, not exactly in the filename. There are at least two ways to deal with this.

---

You can use rename -d. This option makes rename act on the filename only. The following command:

find /c.0/b -depth -exec rename -d s/c/X/ {} \;

will successfully change /c.0/b/c.1/c.2/d into /c.0/b/X.1/X.2/d (so even c.0 is not a problem).

-d is a relatively new option. In general your rename may not support it. In such case use -execdir instead of -exec:

find /c.0/b -depth -execdir rename s/c/X/ {} \;

^{(In general your find may or may not support -execdir.)}

It works because when processing d, the string rename processes is now ./d, not /c.0/b/c.1/c.2/d; similarly when processing c.2, it's ./c.2, not /c.0/b/c.1/c.2. The tool works in the respective parent directory each time, so the relative paths work. By not including the full path in the string, we get the behavior of rename -d without -d.

Well, almost. If you want rename to do something with dots (e.g. change c.2 into c-2 etc.) then the leading dot in ./d or ./c.2 is something you need to deal with. With -d there is no such problem, regardless if you use -exec or -execdir; so use rename -d if you can.

---

Final note

The command in question includes -depth and rename -d. The OP explicitly stated:

I added -depth option later […]

so probably the lack of -depth was the initial problem. But then:

[adding -detph] did not help

Considering this and noting the error (Can't rename …) comes from rename, I conclude -d was also not always there or I'm missing something. For now I think find -depth along with rename -d should be robust. Assuming it really failed for the OP, I cannot explain this (yet).