How to redirect stderr of python3 script using tee pipe?

Question

I am a Debian and RedHat user. I would like to redirect stderr (module logging) from a Python3 script. The script outputs a lot of things and the part I want to capture can be fetched using:

python3 ./script.py --input ./*.txt --verbose 2>> ./script.log

This appends stderr to ./script.log. But I would keep this dialog also in terminal. Usually I achieve this by piping the tee command. The problem is, the following shell line (executed from a bash file):

pyhton3 ./script.py --input ./*.txt --verbose | tee -a ./script.log

Does output into the terminal but does not output anything to ./script.log. Any idea how to proceed?

Answer 1

To display both stdout and stderr on the terminal while only capturing stderr to file, use:

python3 ./script.py --input ./*.txt --verbose 2>&1 1>/dev/tty | tee -a script.log

The way that the shell handles redirections is quite subtle. Here, the 2>&1 results in the command's stderr being piped to the tee command. 1>/dev/tty results in the command's stdout going directly to terminal. Order is important. If the order of these redirections is reversed, nothing would go to the pipe. Alternatively, if 1>/dev/tty were omitted, then both stdout and stderr would be piped to the tee command.

Answer 2

You can redirect your stderr (2) to stdout (1) and then pipe it to tee like that

python3 ./script.py --input ./*.txt --verbose 2>&1 | tee -a ./script.log