I noticed Windows 11 doesn't let ICMP packets through by default. The network needs to be assigned to a non public network and the appropriate firewall rules have to be activated. I found the following firewall rules (in my case it's IP4 in a private network):
Core Networking Diagnostics - ICMP Echo Request (ICMPv4-In)
File and Printer Sharing - ICMP Echo Request (ICMPv4-In)
Activating either of those rules allows ping requests to go through. The description for both rules is the same so why the different names? Does it matter which one I activate?