Why does incoming SSH on macOS Monterey stop working but is fixed temporarily by restarting sshd?

Question

I have been allowing incoming ssh on one of my home machines for many years without incident. Recently it stopped working. I have discovered that it can be fixed by restarting the sshd process, using these commands:

sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist

After this, it will work for a while, but is usually wedged again within 24 hours. I have looked for clues in the logs but so far have not found anything.

Mac mini (late 2014), MacOS Monterey 12.7.3, SSH-2.0-OpenSSH_8.6

Several months ago I installed Cisco AnyConnect VPN, I wondered if that might be the culprit (although I can't be sure that the failures started then, because I had not used remote ssh for quite some time before that), however the behavior is the same after I shut it down, and that application is configured to allow local LAN access even when the VPN is running.

My home LAN a TP-Link router (Archer A7, AC 1750) and have set up DDNS so that I can connect remotely to the computer in question, I first noticed the problem when I was unable to login remotely, and for a while I mistakenly thought the problem was with the router and/or DDNS, but I think the fact that the problem is fixed by restarting sshd on the mac refutes any explanation of that sort.

The sshd version is OpenSSH_8.6p1, LibreSSL 3.3.6. I have Homebrew installed, am using many libraries in my own software, and update fairly regularly.

The problem appears to be something on the Mac itself: I can connect from itself using ssh localhost, but not using the local 192.168.0.x LAN address. The fact that I can connect using ssh localhost I think shows that the ssh-agent is up and running and correctly launching sshd.

I tried disabling the Mac firewall, but it made no difference. (It was already configured to allow incoming ssh).

A few weeks ago I posted version of this question on Stack Overflow, and received a few suggestions before it was disabled because it was "not about programming". But here are a few answers to questions raised in the suggestions:

• The Mac does not have a static IP (that is why DDNS is needed!). However, the router DHCP is configured to always assign the same LAN addresses to my home computers and printer, so the local LAN address does not change. The external WAN address changes only rarely, and that is handled transparently by the DDNS. That does not seem to be an issue because when it stops working, it stops working even when the local LAN address is used from the machine itself (although it works if localhost is specified).

• When ssh is run, it sits there for a long time and eventually prints "operation timed out":

  (base) bash-3.2$ ssh 192.168.0.132
  ssh: connect to host 192.168.0.132 port 22: Operation timed out
  

  When run in fully verbose mode, here is what is displayed before it hangs:

  (base) bash-3.2$ ssh -vvv 192.168.0.132
  OpenSSH_8.6p1, LibreSSL 3.3.6
  debug1: Reading configuration data /Users/jeff/.ssh/config
  debug1: Reading configuration data /etc/ssh/ssh_config
  debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
  debug1: /etc/ssh/ssh_config line 54: Applying options for *
  debug1: /etc/ssh/ssh_config line 58: Applying options for *
  debug2: resolve_canonicalize: hostname 192.168.0.132 is address
  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/jeff/.ssh/known_hosts'
  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/jeff/.ssh/known_hosts2'
  debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
  debug3: ssh_connect_direct: entering
  debug1: Connecting to 192.168.0.132 [192.168.0.132] port 22.
  debug3: set_sock_tos: set socket 3 IP_TOS 0x48
  

At the suggestion of harrymc, I tried ssh -o IPQoS=none <hostname> but the result was the same.

giacomo1968 suggested that it might be related to this issue involving homebrew and $PATH, but I don't think so. For one thing, that issue is about failure to authenticate using keys, whereas in my case the connection attempt just hangs. I did check where openssl was coming from in my shell, and to my surprise it was coming from an anaconda directory, so I edited my rc files commenting out the lines that added the anaconda bin directory to the path, but it made no difference. Furthermore, the problem is exhibited when trying to connect from other hosts where anaconda is not installed (including a terminal emulator on my phone, which has no HomeBrew). I am not sure how to check, but I would assume that sshd and ssh-agent which are run as root are using the system-provided versions.