tools to start many background processes and close cleanly on sigint

Question

I have an ad-hoc case where I as a user want to start a handful of service processes and also be able to interrupt them all together for clean and easy start-stops. The behavior of "docker-compose up" is close to what I want, all the stdout/stderr feeds get to stdout/stderr of my invoking shell, and sigint shuts down all the processes together.

I tried supervisord and circus, but both are a little too production-focused, requiring work arounds to join standard streams, to stay in the foreground, generally not behaving like a foreground process. I'm sure there are decent tools to do this, I've seen it done in plain shell which I would prefer to avoid, but its an option if the boilerplate isn't too extreme.

this result came up on google "overlord" https://github.com/dpedu/overlord which talks about the same kind of goals and troubles, but I don't see any programs here

for common language, say I want to run foo, bar, and baz, merge their stdout/error, and sleep until a sigint which should send sigint to each of the three programs, then wait for each one to exit. again, if I have to do it in shell that's fine, but I'd really like to find out about developer process composition tools if any are built for this kind of task, other than docker-compose,docker,init,or container focused

extra bonus feature would be to preserve the tty info so automatic color would get enabled by subprocesses but no big deal

#!/bin/sh
./foo &
./bar &
./baz &

---

in writing up this post, I ran into

• https://github.com/nicolas-van/multirun

• https://github.com/Yelp/dumb-init

  dumb init is actually almost exactly what I was hoping to find, its already packaged in debian, and its config file is a light shell file

  #!/usr/bin/dumb-init /bin/sh
  ./foo &  # launch a process in the background
  ./bar &
  ./baz  # launch another process in the foreground
  

Answer 1

dumb-init

https://github.com/Yelp/dumb-init

• packaged in debian (apt install dumb-init)

configuration example, create "./start" and give it the content:

#!/usr/bin/dumb-init /bin/sh
./foo &  # launch a process in the background
./bar &
./baz  # launch another process in the foreground

dont forget to chmod +x ./start then run/exit with ./start and ^C as normal

Answer 2

You might be able to use a systemd solution, with foo.service, bar.service and baz.service, all launched by a larger super.service and stopped by that same thing.

Some advantages in doing this:

• You can choose how the set of units responds to failures.
• You don't have to dedicate a terminal window to this (you can close the terminal without affecting the processes)
• Logging can be saved, so you can use tools to parse interesting parts instead of manually scrolling through your terminal.

It doesn't keep these processes in the foreground like you wanted, but unless these are interactive processes, there isn't much value in keeping them in the foreground.

Here is the set of files that would accomplish this:

# ~/.config/systemd/user/super.service
[Unit]
Description=running everything

[Service]
Type=oneshot
ExecStart=/bin/true
RemainAfterExit=yes
--------------------------------------------
# ~/.config/systemd/user/foo.service
[Unit]
PartOf=super.service

[Service]
ExecStart=%h/bin/foo

[Install]
WantedBy=super.service
--------------------------------------------
# ~/.config/systemd/user/bar.service
[Unit]
PartOf=super.service

[Service]
ExecStart=%h/bin/bar

[Install]
WantedBy=super.service
--------------------------------------------
# ~/.config/systemd/user/baz.service
[Unit]
PartOf=super.service

[Service]
ExecStart=%h/bin/baz

[Install]
WantedBy=super.service

Explanation:

• WantedBy=super.service means when super.service is started, this will start too.
• PartOf=super.service means when super.service is stopped, this will stop too.

To control:

• Add each unit to the super group with systemctl --user enable foo bar baz
• start all processes with systemctl --user start super
• stop all processes with systemctl --user stop super
• View the stdout of all processes in the same window with: journalctl -u foo -u bar -u baz. Add -f to continuously follow the stdout on your screen.

Other notes:

• Add KillSignal=SIGINT to any services which only stop gracefully on SIGINT (otherwise they'll get SIGTERM)
• If foo crashes or fails to start, you can force the other processes to stop. Do this by changing WantedBy= to RequiredBy=. This will cause super.service to stop if foo fails to start, causing bar and baz to stop.
• You can also add a BindsTo= relationship to cause super.service to stop if foo bar or baz stops for any reason (such as naturally ending, or failing).
• One nice thing about using journalctl is that it tells you meta-data such as when units were started/stopped and which process printed the message. However, if you don't like this detail, add StandardOutput=append:%h/log to each [Service] section. This will put all of the logs in ~/log. You can then tail -f ~/log to watch it. I think colours may also be preserved better when you do this.

Answer 3

You mention that you prefer to avoid shell, but this is exactly what shells are for, isn't it?

In bash, to start many background processes, you just do

./job1 & ./job2 & ./job3 &

To send an INT signal to all of them:

kill -INT $(jobs -p)

To wait for all of them to be done:

wait

If the issue is that you want to start the same set of jobs repeatedly, you can set

alias start="./job1 & ./job2 & ./job3 &"
alias stop='kill -INT $(jobs -p); wait'

Then just type start to fire them all off, and stop if you want to kill them all.