[HOWTO] OpnSense under virtualisation (Proxmox et.al.)

[meyergru]

You must be very careful with dual-homed hosts:

a. they should not route packets between interfaces
b. that includes setting the gateway on the correct interface

Normally, you use this only in order to be able to reach the machine via a second "leg". I do that sometimes, when I have a VM that lies behind a reverse proxy with a "LAN" leg and I still have a direct IPv6 connection. In such cases, the LAN side has no gateway at all, because the reverse proxy accesses it via its own LAN IP.

[snulgy]

Thanks for the prompt answers meyergru and nero355.
I am well aware about the pitfalls of dual homed hosts, and I did of course double check those settings as I mentioned - packet forwarding is off for IPv4 & 6 (I actually disabled v6 entirely for now to rule out issues), there are only 3 routes which are correct (to each local network on the right interface, plus the default gateway which again points to the right interface). It does smell like asymmetric routing but I haven't yet figured out how this can possibly happen here.
I suspect it's somehow something wrong either at the proxmox ethernet bridge (somehow opnsense picks up packets not destined for its MAC) or maybe it's somewhere else in my small homelab network that something funny happens (it's just a few random inexpensive managed switches from different brands cobbled together).
when I find more time I'll look closer at some pcaps...