Hi,
I use the same type of configuration at work without any major issues, but this is the first time I've encountered a problem with my personal use.
If anyone has ever encountered a similar issue, I would be very grateful for your help.
I have set up a Proxmox cluster with 2 nodes and 1 Qdevice.
On a Synology NAS, I created an NFS share that I mount in the cluster.
Everything is behind OPNSense, the NAS is in one VLAN, and the cluster is in another.
Let's say 192.168.x.0/24 for the cluster and 192.168.y.0/24 for the NAS.
The NFS share mounts in the cluster and appears in each node.
The problem is that despite an explicit rule that either allows everything between the cluster subnet and the NAS, or specifies port 2049.
Or by specifying each IP on both sides.
Once every two or three times at random, OPNSense blocks port 2049, indicating "Default deny / state violation rule."
I should point out that I disabled fragmentation, which did not help, and then I disabled interface cleanup, with the same result.
OPNSense business 25.10_2
Proxmox 9 and 8.4
DS923 DSM up to date
Any suggestions are welcome.
Does any of these two systems have a second network connection to the respective "other" network?
Yes,
The NAS has two interfaces, one in each network, and the firewall restricts access as desired.
NFS is only allowed on LAN 1.
In addition to the management interface, PVE 1 has an interface dedicated to VMs with different tagged VLANs.
PVE2 only has the management interface configured at this time.
Perhaps this quick diagram will make it clearer.
Thank you for your interest.
You have a case of asymmetrical routing. The NAS is answering to Proxmox via its directly connected network.
Check with tcpdump if the NAS has shell access. Or if you check on proxmox then use "tcpdump -n -i <interface> -e" to also show the MAC addresses involved, then compare if the packets come from the OPNsense or the NAS interfaces.
Routing does not work the way you possibly assume it does. A host will always send a reply packet via the shortest path, not "the same way the request arrived". Once a packet has arrived on a host there is no information anywhere in the stack which way it came. Routing is based strictly on destination address.
Thank you very much.
Indeed, the NAS responds from its LAN 2 interface (I can see this from the MAC address in the tcpdump response) but with the IP address of LAN 1 destined for the Proxmox interface.
I admit that this confuses me, as it does not work at all as I had imagined.
And now, the crucial question.
Is there no way to achieve this?
The NAS has two interfaces. Is it not possible to limit responses to just one of its interfaces?
Thank you again, Mr. Hausen.
I actually succeeded.
For the record, just in case:
I enabled multi-gateway on Synology and set static routes for each interface.
Your mention of asymmetric routing helped me find the resources I needed to understand and modify the configuration.
I am relieved.
Best regards and have a nice day.
Great you found it. I am not familiar with Synology. Surprised it can do policy routing. TrueNAS cannot ;-)