I have a C++ application test that creates 10,000 files in an NFS mounted directory, but my test recently failed once due to one file appearing twice with the same name in that directory with all the other 10,000 files. This can be seen on either Linux Centos v4 or v5 where the directory is NFS mounted, but not on the host machine where the disk resides.
How is it even possible to have two files with the same name in the same directory?
[centos4x32 destination] ls -al ./testfile03373
-rwx------ 1 user root 3373 Sep 3 03:23 ./testfile03373*
[centos4x32 destination] ls -al ./testfile03373*
-rwx------ 1 user root 3373 Sep 3 03:23 ./testfile03373*
-rwx------ 1 user root 3373 Sep 3 03:23 ./testfile03373*
[centos4x32 destination] ls -al *testfile03373
-rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
-rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
[centos4x32 destination] ls -alb test*file03373
-rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
-rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
Running the Perl script suggested in one of the answers below:
ls -la *03373* | perl -e 'while(<>){chomp();while(/(.)/g){$c=$1;if($c=~/[!-~]/){print("$c");}else{printf("\\x%.2x",ord($c));}}print("\n");}'
gives:
-rwx------\x20\x201\x20user\x20root\x203373\x20Sep\x20\x203\x2003:23\x20testfile03373*
-rwx------\x20\x201\x20user\x20root\x203373\x20Sep\x20\x203\x2003:23\x20testfile03373*
Printing with the inode (-i) values shows the two copies have the same inode entry (36733444):
[h3-centos4x32 destination] ls -alib te*stfile03373
36733444 -rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
36733444 -rwx------ 1 user root 3373 Sep 3 03:23 testfile03373*
It would seem the directory entry is corrupted somehow.
Could my application have legitimately created this situation or is this a bug in the operating system? Is there anything I can do to protect against this in my program that creates the files?
I'm thinking there is some kind of bug in the NFS mounting software. Also 'umount' and then 'mount' of the NFS drive that has the issue does not resolve it, the repeated entry remains after remount.
Update 1: I've now hit this issue a second time, a few hours later, and the really strange thing is it happened on the exact same file, testfile03373
, although it got a different inode this time, 213352984, for the doubled files. I'll also add that the file is being created on the Centos 5 machine where disk is being hosted, so it is being created locally, and showing correct locally, but all the other machines that NFS mount it are seeing the doubled entry.
Update 2: I mounted the drive on a Centos v6 machine and found the following in /var/log/messages
after listing and seeing the double entry there:
[root@c6x64 double3373file]# ls -laiB testfile03373* ; tail -3 /var/log/messages
36733444 -rwx------. 1 user root 3373 Sep 3 03:23 testfile03373
36733444 -rwx------. 1 user root 3373 Sep 3 03:23 testfile03373
...
Sep 4 14:59:46 c6x64 kernel: NFS: directory user/double3373file contains a readdir loop.Please contact your server vendor. The file: testfile03373 has duplicate cookie 7675190874049154909
Sep 4 14:59:46 c6x64 kernel: NFS: directory user/double3373file contains a readdir loop.Please contact your server vendor. The file: testfile03373 has duplicate cookie 7675190874049154909
Additionally, I found that renaming the file causes the double entry to disappear, but renaming it back causes it to reappear doubled, or alternatively, just touching a new file with the name testfile03373
, causes a double entry to appear, but this only happens in the two directories where this double entry has been seen.