5

How can I list files recursively with the nu shell?

More specifically: All files, folders and everything else in a specified folder and every inner folder, recursively.

  1. I tried ls **, but that does not work.
  2. I googled, found ls **/**.rs on Coming from Bash and tried ls **/*. That seemed to work, but I had to learn the hard way that it lists only some files, not all. I don't know why. But I found an example where ls **/* | where name =~ 'xxx' found less than ls **/*xxx*. (I thought about including my example here, to show what is listed and what not. But the folder structure is too huge.)
  3. I cannot use ls **/*xxx* as I do not want to filter by name every time. Sometimes I need to filter only by other columns.

(I hope the solution is OS independent. After all, the first advantage of nu praised on its homepage is: "Nu works on Linux, macOS, and Windows. Learn it once, then use it anywhere.")

Improve this question
9
  • 1
    Thanks for reposting here. And yes, the solution should be platform independent, but there are definitely platform specific bugs (although I don't know that this is one), so I like to try to reproduce under the OS in question when possible.
    – NotTheDr01ds
    Commented May 13, 2022 at 19:39
  • 1
    Just to confirm, this isn't a case of the file being hidden (or in a hidden directory), right? Does ls -a **/* show any additional files that are missing from the ls **/*?
    – NotTheDr01ds
    Commented May 13, 2022 at 19:54
  • Good point! ls -a **/* does indeed show more files than ls **/*. ls **/* | where name =~ 'xxx' shows the least number of files, ls **/*xxx* shows more and ls -a **/* | where name =~ 'xxx' shows the most files. In my example, ls **/*xxx* lists files in folders where one of the enclosing folders has a name that starts with a dot, but ls **/* does not. Adding the *xxx* constraint therefore increases the number of matches. ls -a **/* | where name =~ 'xxx' additionally lists every file in a directory whose name contains "xxx".
    – user194860
    Commented May 13, 2022 at 20:30
  • Ok, that's interesting -- For me, ls **/*xxx* ignores files in hidden directories (Linux/WSL2). Also, just tried on Windows Nushell -- This is definitely an area where OS-specific behavior is present. On Windows, directories and files that start with a dot are displayed by default with just a normal ls. Those with the hidden "attribute", however, are hidden from ls unless the -a option is used.
    – NotTheDr01ds
    Commented May 13, 2022 at 20:57
  • Oh, and to get it to ignore files in directories that match (but still display the directory name if it matches), ls -a **/* | where ($it.name | path basename) =~ 'xxx'
    – NotTheDr01ds
    Commented May 13, 2022 at 21:00

3 Answers 3

Reset to default
3

I think we've determined at least two things in working through this in comments and chat:

  • ls -a should be used to make sure that files in hidden directories are included in the result.

  • There appears to currently be a bug in Nushell so that ls **/*abc* sometimes will return files in hidden directories that do not (oddly) match "abc" but will ignore files in hidden directories that do match. If anything, I'd expect the results to be reversed, but I'd really expect ls to always ignore files in hidden directories unless the -a flag is used.

    I've written up a Github issue on the topic.

So currently, to reliably return all files with "xxx" in the filename or path, use ls -a **/*xxx*.

If you want to return only files with "xxx" in the filename (but not any other part of the path), then:

ls -a **/*xxx* | where ($it.name | path basename) =~ 'xxx'

Note that the experimental Nushell glob command (in 0.61) will always return files in hidden directories using glob **/*xxx*, but it only returns filenames, and not in a Nushell column like ls, so it doesn't sound like it would work for your use case:

Sometimes I need to filter only by other columns

The same goes for ^find (not the built-in, but the binary).

However, please do be aware that using ls to return filenames for processing does have some limitations, at least with the present Nushell command:

  • There is currently no way to restrict results to a single filesystem, as with the ^find -xdev option.
  • Nushell's ls **/* will recursively follow symlinks. If you have something like ln -s .. parent, then ls will enter an infinite loop. find, by default, does not follow symlinks.

In general, I'd be very careful using ls **/... globs in current Nushell releases. The ^find binary is probably a safer option, at least for now.

Improve this answer
5
  • It's strange this is the accepted answer as it does NOT cleanly answer the question: "How can I list files recursively with the nu shell?", but only gives convoluted examples with all kinds of filters and exceptions
    – reinier
    Commented Mar 2, 2023 at 12:56
  • @reinier I tend to answer the question asked in the body rather than just the "summary" or "title". The normal ways of recursively listing the files with nushell are covered in the question itself. My answer covers why these weren't working for the OP's specific use-case. Since it properly explained the observed behavior, the OP accepted ....
    – NotTheDr01ds
    Commented Mar 2, 2023 at 13:37
  • I found this page as I had the exact same question: How to list files recursively as ls -r doesn't exist. Stackoverflow works best if I go to the accepted answer and just read the solution without needing to find it hidden in the question itself somewhere. If I want to understand the why, and dive deep, it's great that more text and explainers are given. With google surfacing this page as the first hit for listing files recursively it might be a good edit to list the intended way to do it, and then explain with more text on why it has issues and possible workarounds....
    – reinier
    Commented Mar 3, 2023 at 19:27
  • 1
    @reinier See my updated answer below (or perhaps above) that address the "general" use-case now that the bug is resolved. Again, for this particular question/answer, we had to work around a bug that existed at the time, along with the OP's actual request (which was more complex than simply "recursively listing files".
    – NotTheDr01ds
    Commented Mar 3, 2023 at 20:12
  • Thats really helpful for probably most people landing on this page. Thanks
    – reinier
    Commented Mar 6, 2023 at 18:04
2

The bug mentioned in my original answer has been resolved, so, to recursively list all (non-hidden) files and subdirectories in all (non-hidden) subdirectories (tested on Nushell 0.76):

ls **/*

To include hidden files:

ls -a **/*

Pattern matching also works correctly. E.g.:

ls **/*.rs

To list recursively list directory names, but not the files within them:

ls **/**
Improve this answer
-1 
find ./*

or install

tree

"Body must be at least 30 characters; you entered 24."

Improve this answer
2
  • find ./* does not display anything for me. And tree is neither a nu command, nor a plugin. I guess you mean the tree program which has nothing to do with the nu shell. Are you sure your answer is for the nu shell and not for example for bash?
    – user194860
    Commented May 14, 2022 at 15:12
  • Note that find is also a built-in Nushell. I'm assuming that you mean the find binary, which would be ^find in Nushell.
    – NotTheDr01ds
    Commented May 14, 2022 at 23:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .