systemctl daemon-reload: show diff between running config and current file

Question

I run the following command :

systemctl status tomcat10.service
#and get the following warning: 
Warning: The unit file, source configuration file or drop-ins of tomcat10.service changed on disk. Run 'systemctl daemon-reload' to reload units.

If I look at the unit file

ls -l /usr/lib/systemd/system/tomcat10.service
-rw-r--r-- 1 root root 1161 Jul  2 12:36 /usr/lib/systemd/system/tomcat10.service

The file has been modified or at least tempered with.

However even if I compare the current running config using some contraption like this:

systemctl cat tomcat10.service | diff /usr/lib/systemd/system/tomcat10.service -
# Warning: tomcat10.service changed on disk, the version systemd has loaded is 
outdated.
# This output shows the current version of the unit's original fragment and drop-in files.
# If fragments or drop-ins were added or removed, they are not properly reflected in this output.
# Run 'systemctl daemon-reload' to reload units.
0a1
> # /usr/lib/systemd/system/tomcat10.service

But in this case I cannot see anything different at all and the message confuses me, am I really comparing running congig with current config ? and if I am nothing as change so why systemd is asking me for a daemon-reload ?
So my question is how can I reliably compare systemctl unit running config with current startup config?
And more broadly, how to know what has change when doing or is about to change when/before typing systemctl daemon-reload.

Answer 1

Your suspicion is correct: systemctl cat always shows the contents of the relevant unit files as they currently are on disk, not as they are currently in systemd’s view of the system. You can see the latter using systemctl show, but that will give you the overall merged properties of a unit, with no indication of where the values come from (defaults, system units, overrides, etc.).

systemctl warns you that the unit files have changed because their last modification time is newer than their modification time when they were loaded; that’s all that systemd compares.

I’m not aware of a simple way to determine what will change with the next daemon-reload. You can figure it out by comparing systemctl show’s output with the current contents of all the corresponding unit files, but that can end up being complicated.

Answer 2

So in kind of a crisis, I spend 4 hours making a python script without dependencies so I can more easily compare running config with usual config for systemctl unit files.
Script is not perfect and will not catch option being move around in the file but it should catch:

• option that value were change in the file
• option that were remove
• also some runtime option sometime have more stuff like timestamp and all and I made no logic to filter that out.
  This is more a fragile tool than a reliable strong solution, can be use with regular user account not root permission needed:

#! /usr/bin/python
import sys
import subprocess
from pathlib import Path

def parse_unit_options(unit_options_arrays):
    unit_dict = {}
    # Strips the newline character
    for line in unit_options_arrays:
        line=line.strip()
        if not line.startswith('#') and not line.startswith('[') and not line == '':
            option_name = line.split('=',1)[0].strip()
            option_value = line.split('=',1)[1].strip()
            if option_name in unit_dict.keys():
                unit_dict[option_name]=unit_dict[option_name]+' '+option_value
            else:
                unit_dict[option_name]=option_value
        #sort each string the same way
        for conf_name, conf_value in unit_dict.items():
            unit_dict[conf_name]= ' '.join( sorted(conf_value.split(' ') ) )
    return unit_dict

def get_unit_file_options(unit_path):
    unit = open(unit_path, 'r')
    unit_lines = unit.readlines()
    unit_options=[]
    for line in unit_lines:
        line=line.strip()
        unit_options.append(line)
    return unit_options

def get_unit_running_options(unit_name):
    unit_options=[]
    for line in subprocess.check_output(unit_running_config_finder+unit_name, shell=True, text=True).split('\n'):
        line=line.strip()
        unit_options.append(line)
    return unit_options

def get_default_unit_options():
    return get_unit_running_options('default')

unit_name=sys.argv[1]
if unit_name == '':
    print("Need one argument as unit name.")
    quit()
unit_unit_file_finder='systemctl show -P FragmentPath '
unit_running_config_finder='systemctl show --all '
print("Trying to compare running config with current file for unit: "+unit_name)
unit_path=subprocess.check_output(unit_unit_file_finder+unit_name, shell=True, text=True).split("\n")[0]
if unit_path == '' or not Path(unit_path).is_file() :
    print("unit file: "+unit_path+ " could not be found, or doesn't exists.")
    quit()
# Get unif file dict for the file , the running config 
# and the default values
file_unit_options = parse_unit_options(get_unit_file_options(unit_path))
running_unit_options = parse_unit_options(get_unit_running_options(unit_name))
default_unit_options = parse_unit_options(get_default_unit_options())
output_header=['option name','config file', 'running config', 'default config']
output_list=[]
#Compare option from file to running and default value
for option_name, option_value in file_unit_options.items():
    new_line=[option_name,option_value]    
    if option_name in running_unit_options.keys(): 
        new_line.append(running_unit_options[option_name])
    else:
        new_line.append("")
    
    if option_name in default_unit_options.keys(): 
        new_line.append(default_unit_options[option_name])
    else:
        new_line.append("")
    if new_line[1] != new_line[2] or new_line[1] != new_line[3]:
        output_list.append(new_line)
# Compare default value with running and try to identify if 
# option where removed from file
# If default option is different than running option and the key is missing from it means it has been remove.
# if the option has change values then it will be catch by the upper test.
for option_name, option_value in default_unit_options.items():
    new_line=[option_name]
    if not option_name in file_unit_options.keys() and  \
       running_unit_options[option_name] != option_value:
       new_line.append("missing value")
       new_line.append(running_unit_options[option_name])
       new_line.append(option_value)
       output_list.append(new_line)
for line in output_list:
    print(output_header[0]+": "+line[0])
    print('\t'+output_header[1]+": "+line[1])
    print('\t'+output_header[2]+": "+line[2])
    print('\t'+output_header[3]+": "+line[3])