6

I am running multiple processes (hundreds), each of which is in python and invoked using:

command = 'start cmd /k call python %s' % (some_py_prog)
os.system(command)

where the /k leaves the cmd window open after execution. This is good to inspect for errors. However, as I call hundreds of jobs, my screen gets cluttered.

How then to make python close its own host cmd window on successful completion only? I need the errored jobs to remain visible.

2
  • Why are you using a Python script to start a shell window that runs a Python script, instead of the Python script importing and running the other Python scripts/functions (as it should be done)?
    – DeepSpace
    Commented Oct 22, 2017 at 11:17
  • Memory management. If I call as many jobs as I have from one python script, the top level namespace grows and calls the OS for more. That's fine, but I have found that this memory is only released when the entire python program completes (even if you use gc.collect(), the OS still marks the space in use). By doing things this way, namespaces are separated and the memory allocated for each is released by the OS as each python sub-program closes. The sub-program uses other programs (by MS and 3rd parties, forced by my work) - this is the only way I've discovered to manage system memory OK.
    – SteveBuk
    Commented Oct 22, 2017 at 11:42

2 Answers 2

7

Short answer: just use /c instead of /k will cause the cmd script to exit as soon as the command terminates, which is what you ask for:

command = 'start cmd /c call python %s' % (some_py_prog)
os.system(command)

But you are stacking some unnecessary cmd.exe here: one for the os.system call, one for the explicit cmd.c. You could simply use the subprocess module:

subprocess.call('python %s' % (some_py_prog), 
    creationflags = subprocess.CREATE_NEW_CONSOLE)

The creation flag is enough to ask the system to start the new process in a new console that will be closed as soon as the command will be terminated. Optionaly, you can add the shell=True option if you need it to pre-process some cmd goodies like io redirection.

(more details in that other answer from mine, specially in eryksun's comment)

1
  • If you want to keep the new console open if there's an error, you can run it as 'python -i %s'. You can also add the -i option to a shebang if you're using the py launcher.
    – Eryk Sun
    Commented Oct 28, 2017 at 23:27
4

Let us first analyze what the two posted Python script lines really do on execution of the Python script. Thanks goes to eryksun for his deep investigation really using Python resulting in a correct description as it can be read below now.

os.system() results in execution of cmd.exe /C in foreground with a console window and halts execution of the Python script until Windows command interpreter terminates. If the Python script itself is executed in a console, the started cmd.exe inherits this console.

This command process starts with start, an internal command of cmd.exe, one more command process with a foreground console window. Fine, but this second command process terminates immediately on finishing execution of the command. That is not so good if you want to see errors output by the executed script or by Python interpreter itself on running the Python script.

So the second command process starts cmd.exe with option /k to keep running this command process inheriting the console created by start and the console window opened after finishing the execution of the specified command.

The second command process runs internal command call which is not necessary at all because python is in real python.exe, a console application and not a batch file. Therefore call is not needed at all.

It is advisable to always specify applications and scripts with complete name of file, i.e. file name + file extension and not just file name only. If the path to executable/script is known and fixed, it should be specified too. This makes the execution of the application/script independent on what is the current directory and the environment variables PATHEXT and PATH.

Python interpreter executes the specified Python script in second console.

The first command process started with os.system() terminates immediately after start finished which occurs already after starting cmd.exe /k while python.exe is interpreting the Python script.

So remaining is the second command process running the Python interpreter with the script. This command process keeps running even after Python interpreter terminated after finishing execution of specified Python script.

So the goal is to terminate also the second command process with the console window once Python interpreter finished the execution of the Python script, but only if no error occurred on execution of the script.

Well, I don't have Python installed at all, but I suppose it exits with a return code greater 0 on an error on execution of a script. Otherwise the exit code is 0 on successful execution of the script.

So it might work to use a command like this:

command = 'start cmd.exe /K python.exe %s ^&^& exit' % (some_py_prog)
os.system(command)

The command line to execute by second command process started with start cmd.exe /K contains two commands now:

  1. Python interpreter python.exe with the script to execute as parameter
    and
  2. the internal command exit whereby this second command should be executed by Windows command interpreter only if exit code of first command is 0 because of operator &&.

See the answer on Single line with multiple commands using Windows batch file for details about single command line with more than one command to execute.

Each ampersand must be escaped here with escape character of Windows command interpreter which is the caret character ^. This is necessary as otherwise && would be interpreted by first command process running start as additional command to run after successful execution of start.

Please note that I don't have Python installed and therefore made just following test from within a command prompt window using a batch file Test.bat with the single command line @echo %~dp0 executed. & exit /B 1.

start cmd.exe /K call Test.bat ^&^& exit

Test.bat is a batch file and not an executable which requires the usage of command call. The started command process with no specific window title keeps open because Test.bat exits with return code 1. The started command process exits itself if I modify in Test.bat the number 1 to 0 at end of the command line.

Of course it is necessary that the script code itself halts script execution on an error detected by the script code on using this solution.

1
  • Thanks Mofi! This worked and allowed me to see what the bad process was doing / identified a bug to fix. I knew there was one (somewhere) as of 16,128 issued jobs only 16,023 came back... :) Thanks & hats off to you!
    – SteveBuk
    Commented Oct 25, 2017 at 15:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.