Really silly question - routing

[mightyi]

I think I’m having one of those weeks where nothing works right!
I am playing around with Docker at the moment to see if there is viable reason to put Plex/Emby into isolated containers. Currently experimenting with networking.

I configured a macvlan network and managed to get it almost working (didn’t enable promiscuous mode on firewall) and I’ve been playing with IPVlan v3 networking now.  No matter what I do, I can’t get internet traffic routing back to the subnet and it’s driving me nuts - I’m sure it’s something simple but I can’t figure it out…I’m hoping someone can help me feel silly!

I have a route setup on the firewall to push any traffic for the docker subnet to the host Ubuntu vm (vm it set as a gateway). I have also added a lan rule on the firewall to allow all traffic from the docker subnet to the internet from the lan interface. I can see in the logs that traffic is being allowed since I added this rule, and I can ping the firewall and all other devices on the lan from the container in the v3 ipvlan network. But if I ping a web address (Google for example), it resolves the name, I see the traffic being allowed in the fw logs….but no I no response back.  From the ubuntu host it works fine.

Am I missing something obvious here?

[meyergru]

Maybe your VM host is set as a gateway, but is your OpnSense aware of additional networks on the LAN interface that the VM host is connected to? You probably have outbound NAT on your OpnSense which translates only your LAN IP range but not anything "behind" it.

[mightyi]

Sir you are a legend! Exactly the thing I’d forgotten! Had to enable hybrid NAT rules, but as soon as I create one for TCP and one for ICMP everything worked. 

Knew it would be something stupid I’d forgotten…. 

[swimboy]

I'm having the exact same problem, except for the fact that I *did* remember to create hybrid NAT rules for the docker IPVLAN networks.

My routing is definitely working, since I can access my docker containers from other devices on my network and vice versa. But the docker containers cannot reach the internet.

Attached is my IPv4 NAT rule. I have an identical one for IPv6, and one set as static for port 500.

What else can I look at?

[meyergru]

Besides the NAT, you also need a rule to allow outgoing traffic. For the LAN interface, there is a default rule that allows the LAN network to go outside, but nothing more. Have you created a firewall rule to allow your docker IPs to "any"? mightyi did this before so all was missing was the NAT rule in that case.

[swimboy]

Nope, I had those rules too. After further investigation, I saw that I had configured my NAT rule on the LAN interface instead of the WAN interface. Fixed that, and everything's working.