Some hints on detaching the child process from the calling one (starting the child process in background).
Suppose you want to start a long task from a CGI-script, that script. That is, the child process should live longer than the CGI-script script execution process.
The classical example from the subprocess module docsdocumentation is:
import subprocess
import sys
# someSome code here
pid = subprocess.Popen([sys.executable, "longtask.py"]) # callCall subprocess
# someSome more code here
The idea here is that you do not want to wait in the line 'call subprocess' until the longtask.py is finished. But it is not clear what happens after the line 'some more code here' from the example.
My target platform was freebsdFreeBSD, but the development was on windowsWindows, so I faced the problem on windowsWindows first.
On windowsWindows (win xpWindows XP), the parent process will not finish until the longtask.py has finished its work. It is not what you want in a CGI-script script. The problem is not specific to Python,Python; in the PHP community the problems are the same.
The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in winWindows API. If you happen to have installed pywin32, you can import the flag from the win32process module, otherwise you should define it yourself:
DETACHED_PROCESS = 0x00000008
pid = subprocess.Popen([sys.executable, "longtask.py"],
creationflags=DETACHED_PROCESS).pid
/* UPD 2015.10.27 @eryksun in a comment below notes, that the semantically correct flag is CREATE_NEW_CONSOLE (0x00000010) */
On freebsdFreeBSD we have another problem: when the parent process is finished, it finishes the child processes as well. And that is not what you want in a CGI-script script either. Some experiments showed that the problem seemed to be in sharing sys.stdout. And the working solution was the following:
pid = subprocess.Popen([sys.executable, "longtask.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
I have not checked the code on other platforms and do not know the reasons of the behaviour on freebsdFreeBSD. If anyone knows, please share your ideas. Googling on starting background processes in Python does not shed any light yet.