OpnSense after three months use

[tverweij]

I use OpnSense now for three months, and generally speaking I am very content with this firewall.
I come from Kerio, which I used for about 25 years.

If I look at the things I encountered, two points need improvement in OpnSense.

1. The Aliases. It works very well , but documentation of a Hosts alias is almost not possible (yes, I know, make an alias for each IP and then one hosts alias - but sorry, too much work and absolutely not workable as the amount of aliasses explodes. I use now URL Table (IPS) as a work around for this problem ....

2. The outbound. As the name says: it won't work for inbound. This really costs me a lot of headaches as a lot of rules I had in Kerio were impossible to implement on OpnSense, because it can not change the source IP for an inbound rule. I finally worked around this by setting up a Windows machine. I do the port forwarding to this machine and in this machine I do the address translation and reroute to the real destination using "netsh interface portproxy". It works like a charm, but it makes things less transparent and shouldn't be neccerary,

Other than those 2 problems, I think you make a really great product.

EDIT, number 3:
Firewall broke for unknown reasons, leaving all LAN segments isolated from the Internet.
The only thing that helped was restoring the Virtual Harddisk from a backup of 4 days ago. I did not restore the virtual hardware, what means that the problem was on the disk.
I never had to restore a firewall before in 25 years, so this sounds like instability in OpnSense.

Edit: restoring the second firewall with the same problem 8 hours later ...

[Patrick M. Hausen]

For inbound connections that should use a real TCP relay, you can use HAproxy.

But possibly I did not quite get your 2. point. Could you provide an example of what you are routing/relaying where? I am pretty confident that can be solved in OPNsense alone.

[cookiemonster]

Number 2 has got me scratching my head too, maybe two problems not one.

[tverweij]

Explanation of Number 2:

I have 3 HyperV machines, two in the same rack and one on another continent.
Each physical machine runs it's own OpnSense firewall and all firewalls are connected to each other using IPSEC tunnels.

To make it simple (in reality, lots of VM's are running), in this picture I have 2 physical machines and 2 VM's, one on each physical machine:

PhM1 has VM1, IP 10.10.10.5 - this machine has an IPSEC tunnel to  phM2, gateway 10.20.10.2
PhM2 has VM2, IP 172.25.0.7 - this machine has an IPSEC tunnel to  phM1, gateway 10.20.10.3


I get a connection from the internet on PhM1 to port 95 from IP x.x.x.x
On the WAN of PhM1, I defined the following NAT rule: ThisFirewall on port 95 Forward to 172.25.0.7 on port 80.
In the routes, I defined that the network 172.25.0.0/24 has to go through gateway 10.20.10.2.

What happens:
1. The forward sees that it should route it to the gateway 10.20.10.2 (so far so good)
2. PhM2 is entered and it sends it to the correct machine (172.25.0.7)(again, works like a charm)
3. The destination machine answers and as the source address is an address on the internet, the data is routed to it's own internet gateway instead of routing it back to the IPSEC tunnel, breaking the connection flow.

Request from x.x.x.x:
x.x.x.x -> WAN-PhM1 -> 10.20.10.2 -> 10.20.10.3 -> 172.25.0.7
Answer from 172.25.0.7:
172.25.0.7 -> WAN-PhM2 -> x.x.x.x

Result: communication is impossible.
When the source address is an address within PhM1, everything works 100% correct, but as soon as the source is an internet address, the communication fails because it is impossible to ensure the the correct route back is followed.

When an Inbound SNAT is possible, I can translate the internet address to a local address of PhM1 and that would solve the problem.

With SNat
Request from x.x.x.x:
x.x.x.x -> WAN-PhM1 -> SNAT change source to y.y.y.y _>10.20.10.2 -> 10.20.10.3 -> 172.25.0.7
Answer from 172.25.0.7:
172.25.0.7 -> 10.20.10.3 -> 10.20.10.2 -> SNAT restore source to x.x.x.x -> WAN-PhM1 -> x.x.x.x

And then the flow is correct and communication works.

I hope this explanation makes things clear.

[Patrick M. Hausen]

When an Inbound SNAT is possible, I can translate the internet address to a local address of PhM1 and that would solve the problem.

You can trivially achieve that with a TCP relay by e.g. HAproxy.

I would need to experiment to find out if port forwarding combined with outbound NAT on the tunnel would work, too. But I suspect it would.

[tverweij]

That looks like about the same as I did now with a Windows instance that takes care of the SNAT ...

[cookiemonster]

I suspect not quite as there is a haproxy plugin for OPN so if introducing it + rules work, then they can coexist in an OPN installation, cutting out a windows machine. If that was of value to you of course.

[tverweij]

No, the value of my course is to make it easy and transparent. An extra Windows machine is not a problem.
Needing to use a proxy in any way is, as SNAT is invented for this.

And as I never, ever used SNAT for outgoing rules, other than the default SNAT on the WAN adapter, I really don't understand that this is missing in OpnSense. I found out that this was missing after I crossed the line of no return and luckely I found a work around, as this was really a point that, if I found out earlier, would have lead me to disregard OpnSense and search for another alternative.

But as said, beside those two points that make my life less easy (as the VM networks regurarly move from one host to another, based on resources), I like the product.

[cookiemonster]

fair enough

[Patrick M. Hausen]

It's not missing. You can define SNAT for inbound connections by creating an outbound NAT rule on the tunnel interface. Which makes sense if you visualise the flow of packets.

The terms outbound and inbound do not apply to to/from the Internet or the private networks. They are applied on the interface level.

A packet originating on your LAN directed to the Internet is inbound on the LAN interface. A packet port forwarded via NAT from the Internet down some tunnel is outbound on the tunnel interface.

[tverweij]

It's not missing. You can define SNAT for inbound connections by creating an outbound NAT rule on the tunnel interface. Which makes sense if you visualise the flow of packets.

Yes I thought of that. And I tried that. And it failed.
As soon as I do that, I see in the live Log that it works, but no data is going through the tunnel anymore.
I searched the forum for this, and found that SNAT is not working on a IPSEC interface / tunnel that uses routing.

[Monviech (Cedrik)]

It's not missing. You can define SNAT for inbound connections by creating an outbound NAT rule on the tunnel interface. Which makes sense if you visualise the flow of packets.

Yes I thought of that. And I tried that. And it failed.
As soon as I do that, I see in the live Log that it works, but no data is going through the tunnel anymore.
I searched the forum for this, and found that SNAT is not working on a IPSEC interface / tunnel that uses routing.

It was true once that SNAT didn't match on VTI, but I just tested it today and its not true anymore.

Test Setup: https://forum.opnsense.org/index.php?topic=36254.msg176819#msg176819
SNAT Test: https://forum.opnsense.org/index.php?topic=36254.msg176976#msg176976

[tverweij]

This means I have to recreate the tunnels using "Connections"?

Because I use the "Tunnel Settings" to create an IPv4 IKEv2 ESP Route based tunnel.

[Monviech (Cedrik)]

Yes I think that's what you'll have to do. I never got it to work with the legacy tunnels either.

But be careful, this setup I described will make the OPNsense "ONLY" work with VTI tunnels. So no other policy based tunnels will work. (At least not with filter rules and NAT rules. That functionality is then tethered to the ipsec interfaces, and the enc0 interface will be stripped of that functionality)

Only use this setup if you only have route based VPN tunnels and don't plan on any policy based ones in the future.

- I on the other hand only use policy based tunnels and don't use VTI at all. So it's a design choice.

EDIT: I also added packet captures to my proof of concept in the post above to have evidence of SNAT indeed working.

[tverweij]

Only use this setup if you only have route based VPN tunnels and don't plan on any policy based ones in the future.

That means I can not use it, as I have other tunnel modes for customer connections.