Should iptables block RST packets for a websocket connection?

Question

I have the following setup for offloading SSL and running home assistant :

• Raspberry pi A (model 3, 10.0.0.21 below) is running apache2, NextCloud and iptables.
• Raspberry pi B (model 1, 10.0.0.69 below) is running home assistant (vanilla installation, no added integrations, only tracks one iPhone running the home assistant companion app)

Traffic from the internet is port-forwarded by my modem to Pi A, where it offloads the SSL and then sends traffic to either local Nextcloud or pi B running home assistant. The configuration I use to offload SSL and forward traffic to home assistant is:

<IfModule mod_ssl.c>
<VirtualHost _default_:8443>
    LogLevel debug
    ErrorLog ${APACHE_LOG_DIR}/ssl_proxy_error.log
    CustomLog ${APACHE_LOG_DIR}/ssl_proxy_access.log combined

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/mysite/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/mysite/privkey.pem

    <Proxy *>
        Allow from localhost
    </Proxy>

    Header add Connection "Upgrade"
    RequestHeader set Connection "Upgrade"

    ProxyPass        / http://10.0.0.69:8123/
    ProxyPassReverse / http://10.0.0.69:8123/

    RewriteEngine on
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule /(.*) ws://10.0.0.69:8123/$1 [P,L]

    RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
    RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}
</VirtualHost>
</IfModule>

In iptables in Pi A, the relevant rules for forwarding traffic to home assistant on Pi B are:

# Generated by xtables-save v1.8.2
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]
:LOG_AND_DROP - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 10.0.0.0/16 -m state --state NEW -m comment --comment "Allow DMZ" -j ACCEPT
-A INPUT -s 109.32.0.0/13 -m state --state NEW -m comment --comment "Allow KPN" -j ACCEPT
...
-A INPUT -s 83.163.0.0/16 -m state --state NEW -m comment --comment "Allow xs4all" -j ACCEPT
-A INPUT -j LOG_AND_DROP
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j LOG_AND_DROP
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -o eth0 -j ACCEPT
-A OUTPUT -j LOG_AND_DROP
-A LOG_AND_DROP -m limit --limit 5/min -j LOG --log-prefix "Source host denied "
-A LOG_AND_DROP -j DROP
COMMIT

Since adding the home assistant pi, I see that iptables on Pi A is blocking a lot of RST packets. These packets are originating from Pi B, the home asstant pi and are logged with the "Source host denied" prefix. I see these logs rougly every 20 seconds, which is probably because of the rate limit in the log and drop rule of iptables. They look like this:

Apr 29 09:48:37 piA kernel: [7261534.851749] Source host denied IN=eth0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:28:00:00:40:00:40:06:26:77 SRC=10.0.0.69 DST=10.0.0.21 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=8123 DPT=43686 WINDOW=0 RES=0x00 RST URGP=0

After a bit of googling, I suspect there can be two reasons for these packets:

1. An ACK/RST flood attack. This feels unlikely because I see no significant other traffic in iptables, and the forwarded ports on are above the 40.000 range, making it an unlikely target. Traffic on the modem is generally low, and I see no significant amount of denies in the iptables log (unless they are drowned by the RST logs and log rate limiting).
2. The websocket connection opened from the home assistant companion app on my phone is being cut off by either my phone, the operator network, the modem or Pi A, somehow causing an ACK to be sent, resulting in an RST from homeassistant.

On pi B: Running sudo tcpdump -nn -c 300 -i enxb827eb0b58da | grep 8123 (grep to only filter traffic to/from port 8123, see iptables log above) I see the following:

13:48:57.514986 IP 10.0.0.69.8123 > 10.0.0.21.44022: Flags [F.], seq 1436924275, ack 2384595600, win 1005, options [nop,nop,TS val 1950188584 ecr 1262794418], length 0
13:48:57.565543 IP 10.0.0.21.44022 > 10.0.0.69.8123: Flags [.], ack 1, win 501, options [nop,nop,TS val 1262869497 ecr 1950188584], length 0
13:48:57.772090 IP 10.0.0.69.8123 > 10.0.0.21.44024: Flags [F.], seq 1197293903, ack 4042210488, win 1002, options [nop,nop,TS val 1950188841 ecr 1262794678], length 0
13:48:57.815521 IP 10.0.0.21.44024 > 10.0.0.69.8123: Flags [.], ack 1, win 500, options [nop,nop,TS val 1262869747 ecr 1950188841], length 0
13:49:20.512688 IP 10.0.0.69.8123 > 10.0.0.21.43830: Flags [P.], seq 930768631:930768633, ack 1366378890, win 1009, options [nop,nop,TS val 1950211581 ecr 1262837177], length 2
13:49:20.513589 IP 10.0.0.21.43830 > 10.0.0.69.8123: Flags [.], ack 2, win 501, options [nop,nop,TS val 1262892444 ecr 1950211581], length 0
13:49:20.913549 IP 10.0.0.21.43830 > 10.0.0.69.8123: Flags [P.], seq 1:7, ack 2, win 501, options [nop,nop,TS val 1262892844 ecr 1950211581], length 6
13:49:20.913722 IP 10.0.0.69.8123 > 10.0.0.21.43830: Flags [.], ack 7, win 1009, options [nop,nop,TS val 1950211982 ecr 1262892844], length 0
13:49:21.715924 IP 10.0.0.21.43990 > 10.0.0.69.8123: Flags [F.], seq 4244003455, ack 2737788983, win 501, options [nop,nop,TS val 1262893647 ecr 1945760001], length 0
13:49:21.716101 IP 10.0.0.69.8123 > 10.0.0.21.43990: Flags [R], seq 2737788983, win 0, length 0
13:49:42.885341 IP 10.0.0.69.8123 > 10.0.0.21.44026: Flags [F.], seq 1290966553, ack 3680514599, win 1002, options [nop,nop,TS val 1950233954 ecr 1262838859], length 0
13:49:42.935887 IP 10.0.0.21.44026 > 10.0.0.69.8123: Flags [.], ack 1, win 500, options [nop,nop,TS val 1262914867 ecr 1950233954], length 0

Here I note the 10.0.0.21.43990 > 10.0.0.69.8123: Flags [F.] (FIN), which is followed by a 10.0.0.69.8123 > 10.0.0.21.43990: Flags [R] (RST). According to what I find on TCP, I would expect a FIN/ACK, not a RST.

My question is:
How do I find the cause for these RST packets, and what should I do with them, block or allow?