Rsyslog forward logs cannot connect Permission Denied

Question

Have configured Rsyslog to ship logs to a remote location through an SSH tunnel.

However rsyslog complains with "Permission denied":

rsyslogd[28412]: cannot connect to 127.0.0.1:10601: Permission denied [v8.2102.0-10.el8 try https://www.rsyslog.com/e/2027 ]

The server is CentOS Stream 8

The tunnel is verified to be up - ss -lntp - and I'm able to send through it with i.e. echo test | nc 127.0.0.1 601, and the test shows up at the remote.

The tunnel is kept up by autossh running as an unprivileged user, thus the local port must be unprivileged (or you get bind [127.0.0.1]:601: Permission denied when setting up the tunnel).

Forwarding is done through the Rsyslog omfwd Forwarding Output Module

# /etc/rsyslog.d/00-forward.conf
# Forward to remote server through an ssh tunnel / autossh
*.info action(type="omfwd"
         queue.Type="LinkedList"
         queue.Filename="forward_queue_1"
         queue.MaxDiskSpace="3m"
         queue.SaveOnshutdown="on"
         action.ResumeRetryCount="-1"
         target="127.0.0.1" port="10601" protocol="tcp"
        )

Answer 1

Rsyslog runs as root, so you should not normally get a permission denied.

Diving into /var/log/messages (or /var/log/syslog if on i.e. Ubuntu) should provide some clues.

grep 10601 /var/log/messages  # search for the port number
... setroubleshoot[29284]: SELinux is preventing /usr/sbin/rsyslogd from name_connect access on the tcp_socket port 10601. For complete SELinux messages run: sealert -l 005c986c-e0f9-481a-b3c6-0b45a9698ccd
... setroubleshoot[31103]: SELinux is preventing /usr/sbin/rsyslogd from name_connect access on the tcp_socket port 10601.#012#012*****  Plugin connect_ports (92.2 confidence) suggests   *********************#012#012If you want to allow /usr/sbin/rsyslogd to connect to network port 10601#012Then you need to modify the port type.#012Do#012# semanage port -a -t PORT_TYPE -p tcp 10601#012    where PORT_TYPE is one of the following: dns_port_t, dnssec_port_t, http_port_t, kerberos_port_t, mysqld_port_t, ocsp_port_t, postgresql_port_t, rsh_port_t, syslog_tls_port_t, syslogd_port_t, wap_wsp_port_t.#012#012*****  Plugin catchall_boolean (7.83 confidence) suggests   ******************#012#012If you want to

And there it is:

rsyslog has been configured to send to a non standard port and SELinux is denying it.

The sollution - if you keep the non-standard port - is to allow the port in SElinux.

Get details from sealert -l 005c986c-e0f9-481a-b3c6-0b45a9698ccd - the UUID is unique to you of course.

If you want to allow /usr/sbin/rsyslogd to connect to network port 10601
Then you need to modify the port type.
Do
# semanage port -a -t PORT_TYPE -p tcp 10601
    where PORT_TYPE is one of the following: dns_port_t, dnssec_port_t, http_port_t, kerberos_port_t, mysqld_port_t, ocsp_port_t, postgresql_port_t, rsh_port_t, syslog_tls_port_t, syslogd_port_t, wap_wsp_port_t.

I declare that my port is a syslogd_port_t

semanage port -a -t syslogd_port_t -p tcp 10601

# Restart rsyslog for good measure
systemctl restart rsyslog  # or "pkill -HUP rsyslog"

Logs are now sent.