How to get persistent audit logs?

Question

I am currently trying to figure out which application is creating a mysterious socket file called "no" in my home directory. It happens only every few weeks, that is why I have setup auditd with the following rule in /etc/audit.d/rules.d/no:

# This is to clear out old rules, so we don't append to them.
-D

# Feel free to add below this line. See auditctl man page
-w /home/philipp/no

Running a few tests like touch /home/philipp/no confirmed this to work. However, the log files are not persistent.

I have just encountered that the file was created apparently yesterday, but the auditd log is gone - it was overwritten with a new log when I booted the machine today, even though the logs are set to "rotating" in the config.

How can I set auditd to keep all logs? I am using Gentoo with systemd, and version 3.0 of audit.

auditd.conf:

#
# This file controls the configuration of the audit daemon
#

local_events = yes
write_logs = yes
log_file = /var/log/audit/audit.log
log_group = root
log_format = ENRICHED
flush = INCREMENTAL_ASYNC
freq = 50
max_log_file = 8
num_logs = 5
priority_boost = 4
name_format = NONE
##name = mydomain
max_log_file_action = KEEP_LOGS
space_left = 75
space_left_action = SYSLOG
verify_email = yes
action_mail_acct = root
admin_space_left = 50
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND
use_libwrap = yes
##tcp_listen_port = 60
tcp_listen_queue = 5
tcp_max_per_addr = 1
##tcp_client_ports = 1024-65535
tcp_client_max_idle = 0
transport = TCP
krb5_principal = auditd
##krb5_key_file = /etc/audit/audit.key
distribute_network = no
q_depth = 400
overflow_action = SYSLOG
max_restarts = 10
plugin_dir = /etc/audit/plugins.d