conntrack expectation on directed broadcast [closed]

Question

I'm testing out a setup that I would reproduce in a production environment if I get it work. What I have:

• A wireless network 192.168.88.0/24
• An IoT device X in the wireless network
• A Teltonika router (their RutOS firmware is an OpenWrt fork) connecting to the wireless network with IP masquerading (192.168.2.0/24) and providing a 4G failover for internet connectivity due to sometimes unreliable wireless network.
• Another IoT device Y in the natted network behind the Teltonika

These IoT devices need to communicate with each other. They discover each other by means of an UDP broadcast from Y, to which X responds, revealing its IP address. Very simple. By default the broadcast address is the local broadcast address 255.255.255.255 but can be configured to directed broadcast, i.e. 192.168.88.255, which is what I have done.

Since the devices are in different subnets I have enabled bc_forwarding in the Teltonika and of course configured the firewall to allow the return traffic to come through.

Using this configuration I can get the UDP broadcast from Y to X to be successfully forwarded with NAT in place. So far, so good.

Unfortunately, the return packet never comes back to Y!

I have verified that X does send the response to the Teltonika's address (due to NAT) normally. I have also verified that without broadcasts the devices find each other by configuring the "broadcast" address to 192.168.88.whatever-X-got-from-DHCP-this-time. And I have temporarily placed them in the same network to ensure that discovery using the local broadcast address works as expected. So, the problem seems to be that a response to a natted directed broadcast isn't getting routed to Y.

Looking at /proc/net/nf_conntrack in the Teltonika I can see:

ipv4    2 udp      17 18 src=192.168.2.236 dst=192.168.88.255 sport=55879 dport=50607 packets=1 bytes=48 [UNREPLIED] src=192.168.88.255 dst=192.168.88.205 sport=50607 dport=55879 packets=0 bytes=0 mark=0 zone=0 use=2

192.168.88.205 is the address of the Teltonika router in the 192.168.88.0 subnet.

Questions:

1. I interpret that nf_conntrack line to mean that conntrack is expecting a response from the same broadcast address it had sent the original packet to (192.168.88.255)?
2. If answer to 1 is no, what does that line them mean?
3. If answer to 1 is yes, shouldn't it be expecting a response from any address in 192.168.88.0/24? After all, the router knows that 192.168.88.255 is a broadcast address, since it's in the same subnet itself, and only enabling bc_forwarding made the packet get routed. Nobody should be sending packets with the broadcast address as the source address, right?

Answer 1

A limited broadcast to 255.255.255.255 is never forwarded across a router. It is only forwarded by switches inside the broadcast domain (usually all connected switches or a VLAN).

A subnet directed broadcast to the subnet address with all host bits set to one, e.g. 192.168.0.255/24, is forwarded like a unicast up to the last-hop gateway. That last hop decides whether to drop it (that is the default), or forward it as a directed broadcast to all connected nodes when so configured (commonly by using an L2 broadcast). Any nodes before the last hop simply don't know whether the destination is unicast or directed broadcast.

Using NAT complicates things.

They discover each other by means of an UDP broadcast from Y, to which X responds, revealing its IP address.

From the source-NAT perspective, your 'behind the router' network is private/inside, and the wireless network is public/outside.

That won't work. While the private-to-public (source NATed) directed broadcast should work, direct responses to the apparent source address (the router's public address) won't return to the original source through the NAT session since the source address doesn't match the original destination.

Even if you made that work somehow, signaling any private address on the application layer won't work either, because the private address is meaningless to and not reachable from the public address.

Attempting a directed broadcast from public to private cannot work either, as the private address isn't routable on the public side.

Q1: Yes. Q3: No - a source-NAT router's session needs to match a reply's source to a prior destination. You can't have any 'nearby' address answering on behalf. There's no special handling of directed broadcasts as they play no role on the public Internet.

Nobody should be sending packets with the broadcast address as the source address, right?

Exactly.

Note that we cannot help you with consumer-grade devices or ones without optional, paid vendor support in any case. Those are explicitly off topic here, see the help center.