Systemd Refuses to Start Redis Server

Question

I have duplicate redis.conf & /etc/systemd/system/redis.service files on my test and prod servers via scp.

Where prod is able to start and run redis with a unix socket. My test server refuses to start redis, with the following error output:

● redis-server.service - Advanced key-value store
     Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sun 2022-07-31 01:42:29 HKT; 27s ago
       Docs: http://redis.io/documentation,
             man:redis-server(1)

redis-server.service: Scheduled restart job, restart counter is at 5.
Stopped Advanced key-value store.
redis-server.service: Start request repeated too quickly.
redis-server.service: Failed with result 'exit-code'.
Failed to start Advanced key-value store.

I have read many threads related to restart counter is at 5, but none of them have been specific to redis.service.

/etc/systemd/system/redis.service contains:

[Unit]
Description=Advanced key-value store
After=network.target
Documentation=http://redis.io/documentation, man:redis-server(1)

[Service]
Type=forking
ExecStart=/usr/bin/redis-server --loglevel verbose /etc/redis/redis.conf
PIDFile=/run/redis/redis-server.pid
TimeoutStopSec=0
Restart=always
User=redis
Group=redis-socket
RuntimeDirectory=redis
RuntimeDirectoryMode=2755

UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWritePaths=-/var/lib/redis
ReadWritePaths=-/var/log/redis
ReadWritePaths=-/var/run/redis


NoNewPrivileges=true
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE
MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX

# redis-server can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

[Install]
WantedBy=multi-user.target
Alias=redis.service

But when I run systemctl restart redis along with tail -f /var/log/redis/redis-server.log

Nothing is written to the log.... nothing.

This log command journalctl -f -u redis-server gave output journalctl -f -u redis-server ... seems redis doesn't like my custom group redis-socket (members redis and a virtuamin server owner called server-owner, necessary to give server owner access to unix socket, works on Prod...previously worked on Test. Better way to config/define the group? EDIT Problem solved. The core problem was that root had somehow become the owner of /etc/redis correcting ownership on that directory to redis:redis got things working again with my custom group as configured below.

-- Logs begin at Sun 2022-07-31 07:37:12 HKT. --
Jul 31 08:55:09 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:55:09 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:55:09 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 5.
Jul 31 08:55:09 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:55:09 test.example.test systemd[1]: redis-server.service: Start request repeated too quickly.
Jul 31 08:55:09 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:55:09 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:55:13 test.example.test systemd[1]: redis-server.service: Start request repeated too quickly.
Jul 31 08:55:13 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:55:13 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: Starting Advanced key-value store...
Jul 31 08:56:36 test.example.test systemd[8330]: redis-server.service: Failed to determine group credentials: No such process
Jul 31 08:56:36 test.example.test systemd[8330]: redis-server.service: Failed at step GROUP spawning /usr/bin/redis-server: No such process
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Control process exited, code=exited, status=216/GROUP
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:36 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 1.
Jul 31 08:56:36 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: Starting Advanced key-value store...
Jul 31 08:56:36 test.example.test systemd[8333]: redis-server.service: Failed to determine group credentials: No such process
Jul 31 08:56:36 test.example.test systemd[8333]: redis-server.service: Failed at step GROUP spawning /usr/bin/redis-server: No such process
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Control process exited, code=exited, status=216/GROUP
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:36 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 2.
Jul 31 08:56:36 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: Starting Advanced key-value store...
Jul 31 08:56:36 test.example.test systemd[8336]: redis-server.service: Failed to determine group credentials: No such process
Jul 31 08:56:36 test.example.test systemd[8336]: redis-server.service: Failed at step GROUP spawning /usr/bin/redis-server: No such process
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Control process exited, code=exited, status=216/GROUP
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:36 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 3.
Jul 31 08:56:36 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:56:36 test.example.test systemd[1]: Starting Advanced key-value store...
Jul 31 08:56:36 test.example.test systemd[8339]: redis-server.service: Failed to determine group credentials: No such process
Jul 31 08:56:36 test.example.test systemd[8339]: redis-server.service: Failed at step GROUP spawning /usr/bin/redis-server: No such process
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Control process exited, code=exited, status=216/GROUP
Jul 31 08:56:36 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:36 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 4.
Jul 31 08:56:37 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:56:37 test.example.test systemd[1]: Starting Advanced key-value store...
Jul 31 08:56:37 test.example.test systemd[8342]: redis-server.service: Failed to determine group credentials: No such process
Jul 31 08:56:37 test.example.test systemd[8342]: redis-server.service: Failed at step GROUP spawning /usr/bin/redis-server: No such process
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Control process exited, code=exited, status=216/GROUP
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:37 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 5.
Jul 31 08:56:37 test.example.test systemd[1]: Stopped Advanced key-value store.
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Start request repeated too quickly.
Jul 31 08:56:37 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:37 test.example.test systemd[1]: Failed to start Advanced key-value store.
Jul 31 08:56:43 test.example.test systemd[1]: redis-server.service: Start request repeated too quickly.
Jul 31 08:56:43 test.example.test systemd[1]: redis-server.service: Failed with result 'exit-code'.
Jul 31 08:56:43 test.example.test systemd[1]: Failed to start Advanced key-value store.

lil' help?

Answer 1

Ok problem solved.. thanks to the help of @mashuptwice and a good night cleep...

Cause of problem:
Me. I work in multiple terminals and accidentally opened /etc/redis.conf in nano in 2 different terminal windows. While I caught part of my error and deleted .redis.conf.swp file that was created, I simply didn't notice, that the original /etc/redis.conf had changed its permissions to be root/root instead of redis/redis.

The tail -f /var/log/redis/redis-server.log threw no useful error mesages at all to indicate the problem above.

Solution:
The journalctl -f -u redis-server provided by @mashuptwice enabled to the permissions errors beginning with my redis-server.service config and eventually leading me to to find the permissions error on /etc/redis.conf.

In a nutshell, the native redis log, does not seem to very useful when troubleshooting, use journalctl -f -u redis-server if have issues with redis.