Wildcard in PATH (environment variable)

Question

I have a bunch of apps installed locally in my home directory. In order for them to be globally available I add them to PATH in .bashrc:

PATH="$PATH:/home/user/apps/app1/bin"
PATH="$PATH:/home/user/apps/app2/bin"
PATH="$PATH:/home/user/apps/appn/bin"

How can I set it up so that I don't have to add each new one? I'm trying this but it's not working:

PATH="$PATH:/home/user/apps/*/bin"

NOTE: I'm aware I can add them with a loop, but I'm also concerned my PATH variable will become too large, I'm wondering if it is possible to wildcard it somehow.

Answer 1

Wildcards will not be expanded in $PATH, no. Per the bash manual, PATH is:

A colon-separated list of directories in which the shell looks for commands

(my emphasis).

Coming from another direction, the Command Search and Execution section of the manual says, in part:

If the name is neither a shell function nor a builtin, and contains no slashes, Bash searches each element of $PATH for a directory containing an executable file by that name.

... (my emphasis) -- which makes no mention of any special processing done on the path elements, only that they are expected to be directories (as-is).

I'm not sure off-hand what the limit is for the size of a bash variable; I suspect it's available memory. PATH doesn't need to be exported, but many people do; if it is exported, it will need to fit along with other environment variables and arguments into getconf ARG_MAX (ref: https://unix.stackexchange.com/a/124422/117549). A large PATH directory should not induce too much of a performance overhead, since bash uses a hash table to remember locations of previously-found commands (per-session).

If you do hit a limit (visual or technical) with adding each individual application directory to your PATH, I would recommend adding one "symlink" directory to your PATH where you then link in the desired executables from the various applications.

Answer 2

While adding /home/user/apps/*/bin to $PATH means search executables in a directory whose path is /home/user/apps/*/bin (a perfectly valid path, there's nothing that prevents the * character¹ from being used in the name of a file or directory on Unix), you can add the directories whose path matches /home/user/apps/*/bin at the time of writing using:

path+=( /home/user/apps/*/bin(N-/) )

In zsh, where the $PATH variable is tied to $path array like in csh, and the -/ glob qualifier can be used to limit the expansion to files of type directory (or symlink to directory), and N for nullglob

You can also issue a typeset -U path to remove duplicates (U for unique).

In fish, where $PATH is exposed as an array and a nullglob-like behaviour is applied for globs used in set:

set PATH $PATH /home/user/apps/*/bin

(not restricting to files of type directory though).

POSIXly, you can always do:

for dir in /home/user/apps/*/bin; do
  [ -d "$dir" ] && PATH="${PATH:+$PATH:}$dir"
done

In any case, if more directories are added to /home/user/apps/ later, they won't automagically appear in $PATH.

In ksh93, you could use disciplines for that magic to happen:

function PATH.get {
  typeset dir
  for dir in ~(N)/home/user/apps/*/bin; do
    [[ :$PATH: = *:"$dir":* || ! -d $dir ]] || PATH=${PATH:+$PATH:}$dir
  done
}

Where the value of $PATH is computed every time it's referenced.

In zsh, you could have it recomputed before each prompt using a precmd() hook, or with bash using $PROMPT_COMMAND.

---

^{¹ or any character other than / and NUL or even non-characters for that matters.}

Answer 3

I've used a few solutions for this:

1. Use find to dynamically generate $PATH

For this, I've used a couple similar but slightly modified approaches. Suppose you have the following structure:

~/apps
├── foo
│   └── bin
├── bar
│   └── bin
└── blah

We can...

a. add existing apps/*/bin/ folders to $PATH

This will search for any bin/ directories in $HOME/apps/* and add them to the $PATH when your shell loads, skipping any directories which don't have a bin/ sub-directory.

#.bashrc

bins=$(
    find "$HOME/apps/" -maxdepth 2 -type d -name 'bin' \
        | tr '\n' ':' \
        | sed 's/:$//'
)
export PATH="$bins:$PATH"

Produces:

/home/user/apps/foo/bin:/home/user/apps/bar/bin:...

b. add any somedir/*/bin/ to $PATH

...regardless of whether or not each folder in somedir/ actually has a bin/ sub-directory in it. That means, for the above directory structure, even apps/blah/bin will be added to the $PATH -- even though it doesn't exist!

This is useful when one of your directories doesn't currently have a bin/ sub-directory, but it very well could in the future, and you don't want to have to reload your shell for it to be added to your $PATH after you create it.

#.bashrc

bins=$(
    find "$HOME/apps/" -maxdepth 1 -type d \
        | sed 's_$_/bin_' \
        | tr '\n' ':' \
        | sed 's/:$//'
)
export PATH="$bins:$PATH"

Produces:

/home/user/apps/foo/bin:/home/user/apps/bar/bin:/home/user/apps/blah/bin:...

There's possibly a ridiculously negligible performance hit since command lookups will briefly try to look in directories that don't exist, and it feels bad adding non-existing directories to my $PATH, but convenience-wise this is my preferred option.

2. Use stow to generate symlinks to your executables

stow was created for exactly this purpose. I'm currently using this setup to store all of my dotfiles in a git repo and copy them to my home directory :)

If you have a directory structure like:

.
├── bar
│   └── bin
│       ├── bar-executable-1
│       └── bar-executable-2
├── bin
└── foo
    └── bin
        ├── foo-executable-1
        └── foo-executable-2

And you want all of the files in bar/bin/* and foo/bin/* to be linked to in bin/, you can run:

cd somedir
stow -v --restow --target bin/ --dir bar/ bin
stow -v --restow --target bin/ --dir foo/ bin

note: I use --restow for pruning, as it'll delete any old symlinks first before re-creating new symlinks

This generates:

somedir
├── bar
│   └── bin
│       ├── bar-executable-1
│       └── bar-executable-2
├── bin
│   ├── bar-executable-1 -> ../bar/bin/bar-executable-1
│   ├── bar-executable-2 -> ../bar/bin/bar-executable-2
│   ├── foo-executable-1 -> ../foo/bin/foo-executable-1
│   └── foo-executable-2 -> ../foo/bin/foo-executable-2
└── foo
    └── bin
        ├── foo-executable-1
        └── foo-executable-2

And then you can simply add somedir/bin/ to your PATH

making stow a little easier to use

If

1. The only directory inside each appN directory is a single bin/ directory and
2. You're okay with moving the compiled bin/ directory outside of your apps/ directory

then you can use a slightly easier stow command:

/home/user
├── Desktop
├── Documents
├── Pictures
├── Videos
├── apps
│   ├── bar
│   │   └── bin
│   │       ├── bar-executable-1
│   │       └── bar-executable-2
│   └── foo
│       └── bin
│           ├── foo-executable-1
│           └── foo-executable-2
└── bin

cd
stow -v --restow --target ./ --dir apps/ foo bar

This basically says Look in the "apps" folder for "packages" named "foo" and "bar", and copy the entire contents of both of those packages to the current directory using symlinks. So if all that you have in apps/foo/ and apps/bar/ is a single bin/ folder, then this generates a single bin/ folder in the current directory with links to all of the files in apps/foo/bin/ and apps/bar/bin/.

If, however, you have lots of other files/folders in apps/foo/ and apps/bar/ besides just a single bin/ directory, then it'll recreate all of those in your home folder as well, which is probably not desirable, and you should stick with the first stow example.

https://www.gnu.org/software/stow/manual/html_node/Installing-Packages.html