What is the difference between source and export?

Question

I am writing a shell script, to read a file which has key=value pair and set those variables as environment variables. But I have a doubt, if I do source file.txt will that set the variables defined in that file as environment variable or I should read the file line by line and set it using export command ?

Is source command in this case different than export?

Answer 1

When you source the file, the assignments will be set but the variables are not exported unless the allexport option has been set. If you want all the variables to be exported, it is much simpler to use allexport and source the file than it is to read the file and use export explicitly. In other words, you should do:

set -a
. file.txt

(I prefer . because it is more portable than source, but source works just fine in bash.)

Note that exporting a variable does not make it an environment variable. It just makes it an environment variable in any subshell.

Answer 2

source (.) vs export (and also some file lock [flock] stuff at the end)

In short

1. source some_script.sh, or the POSIX-compliant equivalent, . some_script.sh, brings variables in from other scripts, while
2. export my_var="something" pushes variables out to other scripts/processes which are called/started from the current script/process.

Using source some_script.sh or . some_script.sh in a Linux shell script is kind of like using import some_module in Python, or #include <some_header_file.h> in C or C++. It brings variables in from the script being sourced.

Using export some_var="something" is kind of like setting that variable locally, so it is available for the rest of the current script or process, and then also passing it in to any and all sub-scripts or processes you may call from this point onward.

More details

So, this:

# export `some_var` so that it is set and available in the current
# script/process, as well as in all sub-scripts or processes which are called
# from the current script/process
export some_var="something"
# call other scripts/processes, passing in `some_var` to them automatically
# since it was just exported above! 
script1.sh  # this script now gets direct access to `some_var`
script2.sh  # as does this one
script3.sh  # and this one

is as though you had done this:

# set this variable for the current script/process only
some_var="something" 
# call other scripts/processes, passing in `some_var` to them **manually**
# so they can use it too 
some_var="something" script1.sh  # manually pass in `some_var` to this script
some_var="something" script2.sh  # manually pass in `some_var` to this script
some_var="something" script3.sh  # manually pass in `some_var` to this script

except that the first version above, where we called export some_var="something" actually has a recursive passing or exporting of variables to sub-processes, so if we call script1.sh from inside our current script/process, then script1.sh will get the exported variables from our current script, and if script1.sh calls script5.sh, and script5.sh calls script10.sh, then both of those scripts as well will get the exported variables automatically. This is in contrast to the manual case above where only those scripts called explicitly with manually-set variables as the scripts are called will get them, so sub-scripts will NOT automatically get any variables from their calling scripts!

How to "un-export" a variable

Note that once you've exported a variable, calling unset on it will "unexport it", like this:

# set and export `some_var` so that sub-processes will receive it
export some_var="something"
script1.sh  # this script automatically receives `some_var`

# unset and un-export `some_var` so that sub-processes will no longer receive it
unset some_var
script1.sh  # this script does NOT automatically receive `some_var`

In summary

1. source or . imports.
2. export exports.
3. unset unexports.

Example

Create this script:

source_and_export.sh:

#!/bin/bash

echo "var1 = $var1"
var2="world"

Then mark it executable:

chmod +x source_and_export.sh

Now here is me running some commands at the terminal to test the source (.) and export commands with this script. Type in the command you see after the lines beginning with $ (not including the comments). The other lines are the output. Run the commands sequentially, one command at a time:

$ echo "$var1"              # var1 contains nothing locally.

$ var1="hello"              # Set var1 to something in the current process 
                            # only.
$ ./source_and_export.sh    # Call a sub-process.
var1 =                      # The sub-process can't see what I just set var1 
                            # to.
$ export var1               # **Export** var1 so sub-processes will receive it.
$ ./source_and_export.sh    # Call a sub-process.
var1 = hello                # Now the sub-process sees what I previously set 
                            # var1 to.
$ echo "$var1 $var2"        # But, I (my terminal) can't see var2 from the 
                            # subprocess/subscript.
hello 
$ . ./source_and_export.sh  # **Source** the sub-script to _import_ its var2 
                            # into the current process.
var1 = hello
$ echo "$var1 $var2"        # Now I CAN see what the subprocess set var2 to 
                            # because I **sourced it!**
hello world                 # BOTH var1 from the current process and var2 from 
                            # the sub-process print in the current process!
$ unset var1                # Unexport (`unset`) var1.
$ echo "$var1"              # var1 is now NOT set in the current process.
$ ./source_and_export.sh    # And the sub-process doesn't receive it either.
var1 = 
$ var1="hey"                # Set var1 again in the current process.
$ . ./source_and_export.sh  # If I **source** the script, it runs in the 
                            # current process, so it CAN see var1 from the 
                            # current process!
var1 = hey                  # Notice it prints.
$ ./source_and_export.sh    # But if I run the script as a sub-process, it can 
                            # NOT see var1 now because it was `unset` 
                            # (unexported) above and has NOT been `export`ed 
                            # again since then!
var1 =                      # So, var1 is not exported to the subprocess.
$

Using files as global variables between processes

Sometimes, when writing scripts to launch programs and things especially, I have come across cases where export doesn't seem to work right. In these cases, sometimes one must resort to using files themselves as global variables to pass information from one program to another. Here is how that can be done. In this example, the existence of the file ~/temp/.do_something functions as an inter-process boolean variable:

# ------------------------------------------------------------------------------
# In program A, if the file "~/temp/.do_something" does NOT exist, 
# then create it
# ------------------------------------------------------------------------------
mkdir -p ~/temp
if [ ! -f ~/temp/.do_something ]; then
    touch ~/temp/.do_something  # create the file
fi


# ------------------------------------------------------------------------------
# In program B, check to see if the file exists, and act accordingly
# ------------------------------------------------------------------------------

mkdir -p ~/temp
DO_SOMETHING="false"
if [ -f ~/temp/.do_something ]; then
    DO_SOMETHING="true"
fi

if [ "$DO_SOMETHING" == "true" ] && [ "$SOME_OTHER_VAR" == "whatever" ]; then 
    # remove this global file "variable" so we don't act on it again
    # until "program A" is called again and re-creates the file
    rm ~/temp/.do_something 
    do_something
else
    do_something_else
fi

Simply checking for the existence of a file, as shown above, works great for globally passing around boolean conditions between programs and processes. However, if you need to pass around more complicated variables, such as strings or numbers, you may need to do this by writing these values into the file. In such cases, you should use the file lock function, flock, to properly ensure inter-process synchronization. It is a type of process-safe (ie: "inter-process") mutex primitive. You can read about it here:

1. The shell script flock command: https://man7.org/linux/man-pages/man1/flock.1.html. See also man flock or man 1 flock.
2. The Linux library C command: https://man7.org/linux/man-pages/man2/flock.2.html. See also man 2 flock. You must #include <sys/file.h> in your C file to use this function.

References

1. Ask Ubuntu: source vs export vs export LD_LIBRARY_PATH
2. My own experimentation and testing.
3. I'll be adding the above example to my project on GitHub here, under the bash folder: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world

See also

1. My personal website article on "How do you write, import, use, and test libraries in Bash?"

   This talks more about sourcing files with the . operator, and how to make the file not run if you source it, by using this fancy Python-like magic:

   if [ "$__name__" = "__main__" ]; then
       main "$@"
   fi
   

2. My answer on Importing functions from a shell script

3. My answer on What is the bash equivalent to Python's if __name__ == '__main__'?