11

I've been troubleshooting an issue with a sysVinit service not coming online properly at boot within a systemd environment. What I've found is that when no service file or overrides are present in /etc/systemd/system/ for the said service, it autostarts properly. In this case, as I understand it, systemd should be dynamically loading the startup script via reading "legacy" sysvinit scripts present on the system, although I'm not 100% clear on that.

What I'm confused about is that as soon as I pass the edit --full option to systemctl for said service, a flat file is generated at /etc/systemd/system/ and said service now fails to autostart at boot. Using the edit option and trying to add any overrides also seems to cause the service to fail to boot.

Examples, if needed, provided below...

Example of the system when it works

The service (on Centos), in this example called "ProgramExample" has an init script placed in /etc/init.d/programexample and also /etc/rc.d/init.d/programexample:

# ls -l /etc/rc.d/init.d/programexample
-rwxr-xr-x. 1 root root 2264 Mar 29 14:11 /etc/rc.d/init.d/programexample

No service file present at /etc/systemd/system/:

# ls -lh /etc/systemd/system/programexample.service
ls: cannot access /etc/systemd/system/programexample.service: No such file or directory

Systemctl status output in this configuration:

# systemctl status programexample.service
● programexample.service - LSB: Start Program Example at boot time
   Loaded: loaded (/etc/rc.d/init.d/programexample; bad; vendor preset: disabled)
   Active: active (exited) since Wed 2017-03-29 15:53:06 CDT; 14min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1297 ExecStart=/etc/rc.d/init.d/programexample start (code=exited, status=0/SUCCESS)

Mar 29 15:53:05 centos7-box systemd[1]: Starting LSB: Start ProgramExample at boot time...
Mar 29 15:53:05 centos7-box su[1307]: (to programexample) root on none
Mar 29 15:53:06 centos7-box programexample[1297]: ProgramExample (user programexample): instance name set to centos7-box
Mar 29 15:53:06 centos7-box programexample[1297]: instance public base uri set to https://192.168.0.148.programexample.net/programexample/
Mar 29 15:53:06 centos7-box programexample[1297]: instance timezone set to US/Central
Mar 29 15:53:06 centos7-box programexample[1297]: starting java services
Mar 29 15:53:06 centos7-box programexample[1297]: ProgEx server started.
Mar 29 15:53:06 centos7-box systemd[1]: Started LSB: Start ProgramExample at boot time.

With the above configuration, without any files created/placed in /etc/systemd/system/, the ProgramExample service autostarts properly.

Once systemctl edit --full (or just edit) is used:

Once any edits are passed to systemctl, I have observed the following:

  • A flat file or an override directory will be placed in /etc/systemd/system/
  • Said service, in this case ProgramExample, fails to start at boot.
  • I will be unable to "enable" said service using systemctl

Systemctl status output in this configuration (post edit):

# systemctl status programexample.service
● programexample.service - LSB: Start ProgramExample at boot time
   Loaded: loaded (/etc/rc.d/init.d/programexample; static; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:systemd-sysv-generator(8)

This is the service file that is being generated and placed in /etc/systemd/system/ when using the edit --full option:

# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/rc.d/init.d/programexample
Description=LSB: Start ProgramExample at boot time
Before=runlevel2.target
Before=runlevel3.target
Before=runlevel4.target
Before=runlevel5.target
Before=shutdown.target
Before=adsm.service
After=all.target
After=network-online.target
After=postgresql-9.4.service
Conflicts=shutdown.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/etc/rc.d/init.d/programexample start
ExecStop=/etc/rc.d/init.d/programexample stop
ExecReload=/etc/rc.d/init.d/programexample reload

What is happening here? Am I correct that without the flat service file and/or service override directory in /etc/systemd/system/ that systemd is dynamically reading this information from said service's init script? I've tried numerous iterations of editing the service file at /etc/systemd/system/ and leaving the default file in place and cannot get autostarting to work or the service to go into an "enabled" state.

I believe it would be preferable to have a systemd .service file for systemd configurations instead of relying on systemd to read from init script LSB headers for compatibility and concurrency reasons but the default file systemd is creating is failing to start or enable along with numerous other more simple iterations of the .service file I've attempted.

2 Answers 2

10

I've now found that the issue was that the service file automatically generated by systemd-sysv-generator lacks an install section with a WantedBy option. I added the following to my generated file at /etc/systemd/system/programexample.service which allowed me to properly enable the service:

[Install]
WantedBy = multi-user.target

After that I ran

systemctl daemon-reload

to ensure my service file was read by systemd.

Now I received a proper notification that my service was actually symlinked somewhere to be "enabled":

[root@centos7-box ~]# systemctl enable programexample.service
Created symlink from /etc/systemd/system/multi-user.target.wants/programexample.service to /etc/systemd/system/programexample.service.

This link helped me better understand the service file.

I am not a fan of the way that systemd-sysv-generator does not include an install section with a WantedBy option by default. If systemd can dynamically read the LSB headers and properly start services at boot, why doesn't it generate the service file accordingly? I suppose some systemd growing pains are to be expected.

Update July 7 2020:

Working with Debian Buster and trying to enable a SysVInit legacy service, I was presented with this wonderful message, which I believe would have saved me some time when I dealt with this issue in 2017:

Synchronizing state of programexample.service with SysV service script with 
/lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable programexample
The unit files have no installation config (WantedBy=, RequiredBy=, Also=,
Alias= settings in the [Install] section, and DefaultInstance= for template units). 
This means they are not meant to be enabled using systemctl. 
Possible reasons for having this kind of units are:

• A unit may be statically enabled by being symlinked from another unit's
  .wants/ or 
.requires/ directory.
• A unit's purpose may be to act as a helper for some other unit which has
  a requirement dependency on it.
• A unit may be started when needed via activation (socket, path, timer,
  D-Bus, udev, scripted systemctl call, ...).
• In case of template units, the unit is meant to be enabled with some
  instance name specified.
6
  • 1
    SystemV doesnt make any service files at /etc/systemd/system/ Commented Nov 20, 2019 at 4:51
  • @PhilipRego You are correct. In the context above, systemd-sysv-generator does this.
    – Toby
    Commented Nov 20, 2019 at 22:22
  • I'm using SYSTEMD-SYSV-GENERATOR (8) and it didn't create a service file there. Commented Nov 21, 2019 at 3:50
  • @PhilipRego This question and answer were written and answered using a vanilla CentOS 7 build (and re-tested again just now on CentOS 7 and Debian 10). You may need to consult documentation for your specific system if you are using a different OS. Assuming you are on CentOS 7 or Debian 9+, are you saving the generated .system file after passing the "systemctl edit --full"? If you are having a hard time locating your service files, you may want to try using a tool such as mlocate and locating your .service file that way.
    – Toby
    Commented Nov 21, 2019 at 16:54
  • This is the folder my script was in /etc/rc.d/init.d/ Commented Dec 1, 2019 at 1:47
0

In my case the issue caused because of 2 different java versions on the system of which one version (Oracle JDK) was installed later the installation on jenkins in my system. I did ran alternatives -config java command and set the java to the open jdk version that got installed along with jenkins. and restarted jenkins server service. That´s it.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .