Messages

I was finally able to resolve the issue. It was related to MTU. site2 WAN is a DS-Lite tunnel and does not support the default MTU of 1,500.

Some more helpful observations:

- switching OpenVPN from UDP to TCP caused connection reset every few minutes
- moving site2 to a different WAN fixed the issue

I still wonder why "Override MTU" is enabled by default... Is there any advantage in ignoring the ISP config without explicitly setting your own? I mean, other auto-configuration options like DHCP are also enabled by default on the WAN interface...

I think there might be some issue with NAT. That is at least the part I don't really understand.

Is there any difference between my site1 OpenVPN <=> WAN config and the default LAN <=> WAN config of OPNsense (or simple consumer routers)? Do I need anything else than "Outbound NAT" to completely hide the local topology?

I need some help from the network experts. I'm trying to create this scenario:

- 2 OPNsense installations (site1 and site2)
- both connected to the internet
- clients on site2 LAN should use the internet connection of site1 for all traffic, not only HTTP(S)
- the servers to which the clients connect should not be able to detect that site2 even exists

This is what I've done so far:

- create a site2site tunnel using this guide: https://docs.opnsense.org/manual/how-tos/sslvpn_instance_s2s.html
- set "Redirect gateway" to "default" on site1 vpn config
- create "IN: allow any to any" firewall rules on site1 OpenVPN, site2 OpenVPN and site2 LAN
- create "OUT: block any destination except site1 public IP" rule on site2 WAN (to prevent accidently revealing site2 public IP)
- enable outbound NAT (hybrid mode) for any source on site1 WAN

Result:

- clients can access most web pages
- ifconfig.me shows public IP of site1

Problem:

- cannot log into Microsoft account
- login form shows up during first try
- when trying to submit the username, the connection drops
- the whole domain login.live.com is no longer accessible, including the login form that was successfully loaded before
- other domains are still accessible
- restarting all systems does not solve the problem (login form still not accessible)
- resetting the browser or using a private window makes the login form accessible again
=> looks like a ban from Microsoft side

Additional note:

- using the web proxy of site1 allows clients to successfully log into the Microsoft account using site1 public IP

Questions:

- How can Microsoft still detect the VPN configuration?
- How can I further debug the problem?

After some more hours of digging and finally setting up another complete environment with different hardware, I was able to track this down: It was working all the time, all I had to do was clear the state table.  >:(

I always started by accessing the second LAN, then adding a blocking rule and expecting access to be lost immediately. Connections that are already established are not touched by new firewall rules. Although perfectly valid, this behavior is quite unintuitive - especially if your old firewall was stateless.

Maybe you could add a hint to this somewhere in the GUI to save other people from these hours of frustration?

OPNsense is running on Hyper-V 2012. The physical server is connected to the switch via 3 NICs, Teaming is enabled in Hyper-V. The team NIC is assigned to the OPNsense VM with trunking enabled, VLAN separation is done by OPNsense.

I installed OPNsense 16.7 a year ago, did just the basic configuration and added the VLANs, then added "allow everything" firewall rules on each VLAN. It has been running since then, I just did all the upgrades from time to time. Now I'm trying to make the "allow everything" rules a bit more secure...

I've been struggling with setting up some basic firewall rules for hours now. It looks like everything I try is ignored. I have created a WAN interface and multiple LAN interfaces. Routing should be done from every LAN to the WAN, but not between the LANs.

I tried to disable ALL firewall rules on EVERY interface and even added a generic "block everything" rule on one LAN. But I can still send ICMP requests and reach an HTTP server on this LAN from another LAN. The only way I found working was to remove the interface's IP address of the LAN with the HTTP server - so the traffic is definitely flowing through OPNsense.

What is going wrong here? Do you have to explicitly enable the firewall somewhere?

[configd_ctl.py]

Hi Franco,

Thanks for the hint to apinger. It is configured to execute a script called configd_ctl.py. Replacing the script with a custom one works fine.

I would prefer to place my custom script outside the opnsense internals, so I have to change the apinger config. Can you tell me where it is generated? Looks like it is overwritten automatically...

I'm afraid I currently don't have the time to rework the gateway monitoring, as that would require digging into the opnsense internals first...

Regards,
mimo

Is there some way to trigger a custom action when opnsense detects a gateway failure?

Following use case:

My backup internet connection is via LTE. I have a small data package with just some 100 MB as default and need to switch to a better one before connecting the whole network via this gateway.