Messages

Wonderful: just upgraded to 24.7.1 and all widgets load without problem. Thanks for your work!

Just tried with the dev console active: all widget load errors seem to be of type "timeout":

Code Select Expand

Failed to load content for widget: systeminformation, Error: Object { xhr: {...}, textStatus: "timeout", errorThrown: "timeout" }

Thank you Franco, and sorry for my outburst. Even after reset the dashboard and restart the webgui, almost all widgets fails to load (see attached screenshot) and even whose loaded crash after I change UI page. In particular I've never be able to see the contents of the "System Information" widget.

I've only tested the new release in a VM with low resources, which worked slowly with the previous release, but this one seems to drain heavily the vCPU, to the point to make the webui unusable.

I'm definitely looking forward the 24.7.1 release. And thanks for your precious work! :)

Just upgraded from 24.1.10_8 to 24.7_9, and the dashboard looks absolutely broken. I sincerely trust OPNsense and its developers, but really I'm asking why to change something that worked well with such a mess! :/

No bug, only a wrong configuration by me. Since I put the wrong subnet mask combination in the BINAT config, which was giving me the same error I obtained by using the Outbound NAT config, I had the impression that both of them didn't work... :(

I solved by following the explanation I read in this page: "NAT before IPSEC" (I didn't understand the concept by reading the official documentation at https://docs.opnsense.org/manual/how-tos/ipsec-s2s-binat.html on the first try).

[OpenVPN clients (10.199.8.0/25) <--> OPNsense (LAN CARP VIP: 10.199.9.126/25) <--> IPsec remote site (192.168.170.0/24)]

Hi to all, I've the following scenario (a two nodes OPNsense HA cluster, serving as both OpenVPN c2s and IPsec s2s terminator):

OpenVPN clients (10.199.8.0/25) <--> OPNsense (LAN CARP VIP: 10.199.9.126/25) <--> IPsec remote site (192.168.170.0/24)

The IPsec remote site only knows the LAN network (10.199.9.0/25), neither I can add 10.199.8.0/25 in phase 2 (no co-operation from the other side, sigh!). For this reason, in accordance to documentation, I did the following tasks:

• added the network 10.199.8.0/25 to "Manual SPD entries" in IPsec tunnel's phase 2 config;
• created a NAT outbound rule to mask net 10.199.8.0/25 with address 10.199.9.126 on IPsec interface
  (nat on enc0 inet from 10.199.8.0/25 to 192.168.170.0/24 -> 10.199.10.126 port 1024:65535)

Now remote OpenVPN clients can connect to IPsec remote site, but about 90% of the traffic is lost, because it looks like that only the 1st and only a few following packets are actually natted to the destination! :(

You can look here after the evidence of the issue. I sent three ICMP echo requests from an OpenVPN client (10.199.8.46) to the host 192.168.170.1. Here is what the destination host sees:

Code Select Expand


17:37:07.691901 IP 10.199.10.126 > 192.168.170.1: ICMP echo request, id 9817, seq 0, length 64
17:37:07.691947 IP 192.168.170.1 > 10.199.10.126: ICMP echo reply, id 9817, seq 0, length 64
17:37:08.699266 IP 10.199.8.46 > 192.168.170.1: ICMP echo request, id 56334, seq 1, length 64
17:37:08.699309 IP 192.168.170.1 > 10.199.8.46: ICMP echo reply, id 56334, seq 1, length 64
17:37:09.700524 IP 10.199.8.46 > 192.168.170.1: ICMP echo request, id 56334, seq 2, length 64
17:37:09.700559 IP 192.168.170.1 > 10.199.8.46: ICMP echo reply, id 56334, seq 2, length 64

As you can read, only the first packet comes from the NAT IP (10.199.10.126), while the others seem to pass the firewall without any translation.

Anyone can help me to understand whether this is a bug, or I did something wrong in configuration?

Thanks for any reply!


As additional info: same problem if I use the private LAN IP of CARP node in that NAT rule, or if I use a BINAT 1:1 rule. I see the NAT translation in states table. All traffic is allowed between OpenVPN clients and IPsec remote site, and from LAN to IPsec remote site.

As you  can read in the GitHub issue page ( https://github.com/opnsense/plugins/issues/1500#issuecomment-584241501), the tinc package built for the new 20.x branch is working. The issue has been closed.

You can follow any progress in the GitHub issue page. In the mean time, it's pretty trivial to work around the problem: install tinc from the official FreeBSD package repository.

I just read this. I opened an issue some time before: https://github.com/opnsense/plugins/issues/1500

I'm still researching about the cause of this.

Yes, it's even simpler: remove VIP, change IP on LAN and re-create VIP just works.
Maybe there is a flag "this interface has a static ip" somewhere in OPNsense's frontend configuration which was not set during first configuration of LAN interface? :)

Wow! I removed CARP vhid on LAN, changed the IP on the LAN interface (a couple of time, to set back the original), then re-created CARP vhid and... voila'! :)

Now this happens when I bring LAN interface down on master, and preemption works like any other vhid:

OPN01 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.56.66 ::)

Thus OPNsense simply didn't "perceived as static" the LAN IP address the first time I set it up? :P

Sure, you can find screenshots in the attached zip file. I also attached full configuration files, thus you can easily reproduce the environment using virtual machines.

Thanks.

[Oct 4 09:57:16 OPN01 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for WAN(wan) but ignoring since interface is configured with static IP (172.21.3.66 :: )]

Thank you very much Michael. In the mean time I added a 4th interface (DMZ) to both nodes and configured a new CARP vhid on them. The preemption works as expected when both WAN or DMZ links go down, only LAN link failure seems to trigger the issue.

Here after the logs from both nodes when the WAN link goes down on master:

--- MASTER NODE
Oct  4 09:57:15 OPN01 kernel: carp: 2@em0: MASTER -> INIT (hardware interface down)
Oct  4 09:57:15 OPN01 kernel: carp: demoted by 240 to 240 (interface down)
Oct  4 09:57:15 OPN01 kernel: em0: link state changed to DOWN
Oct  4 09:57:15 OPN01 kernel: carp: 3@em2: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:57:15 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em2: 3
Oct  4 09:57:15 OPN01 kernel: carp: 1@em1: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:57:15 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em1: 3
Oct  4 09:57:16 OPN01 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for WAN(wan) but ignoring since interface is configured with static IP (172.21.3.66 :: )
Oct  4 09:57:16 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "BACKUP" for vhid 3
Oct  4 09:57:16 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.56.254 - LAN-CARP (1@em1)" has resumed the state "BACKUP" for vhid 1

--- BACKUP NODE
Oct  4 09:57:15 OPN02 kernel: carp: 3@em2: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:57:15 OPN02 kernel: carp: 1@em1: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:57:15 OPN02 kernel: arp: 192.168.56.254 moved from 00:00:5e:00:01:01 to 08:00:27:c3:73:2b on em1
Oct  4 09:57:16 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "MASTER" for vhid 3
Oct  4 09:57:16 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.56.254 - LAN-CARP (1@em1)" has resumed the state "MASTER" for vhid 1
Oct  4 09:57:19 OPN02 kernel: carp: 2@em0: BACKUP -> MASTER (master timed out)
Oct  4 09:57:19 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "172.21.3.65 - WAN-CARP-01 (2@em0)" has resumed the state "MASTER" for vhid 2


And here are the logs when I test the link failure on LAN interface:

--- MASTER NODE
Oct  4 09:53:24 OPN01 kernel: carp: 1@em1: MASTER -> INIT (hardware interface down)
Oct  4 09:53:24 OPN01 kernel: carp: demoted by 240 to 240 (interface down)
Oct  4 09:53:24 OPN01 kernel: em1: link state changed to DOWN
Oct  4 09:53:24 OPN01 kernel: carp: 3@em2: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:53:24 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em2: 3
Oct  4 09:53:24 OPN01 kernel: carp: 2@em0: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:53:24 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em0: 3
Oct  4 09:53:24 OPN01 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for lan
Oct  4 09:53:24 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em1: 3
Oct  4 09:53:24 OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em1: 3
Oct  4 09:53:24 OPN01 kernel: carp: demoted by -240 to 0 (vhid removed)
Oct  4 09:53:24 OPN01 kernel: em1: promiscuous mode disabled
Oct  4 09:53:25 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "BACKUP" for vhid 3
Oct  4 09:53:25 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "172.21.3.65 - WAN-CARP-01 (2@em0)" has resumed the state "BACKUP" for vhid 2
Oct  4 09:53:25 OPN01 kernel: carp: 3@em2: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:53:25 OPN01 kernel: carp: 2@em0: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:53:25 OPN01 kernel: arp: 172.21.3.65 moved from 00:00:5e:00:01:02 to 08:00:27:3b:3b:ad on em0
Oct  4 09:53:26 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "MASTER" for vhid 3
Oct  4 09:53:26 OPN01 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "172.21.3.65 - WAN-CARP-01 (2@em0)" has resumed the state "MASTER" for vhid 2


--- BACKUP NODE
Oct  4 09:53:24 OPN02 kernel: carp: 3@em2: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:53:24 OPN02 kernel: carp: 2@em0: BACKUP -> MASTER (preempting a slower master)
Oct  4 09:53:24 OPN02 kernel: arp: 172.21.3.65 moved from 00:00:5e:00:01:02 to 08:00:27:db:85:ef on em0
Oct  4 09:53:24 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "MASTER" for vhid 3
Oct  4 09:53:25 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "172.21.3.65 - WAN-CARP-01 (2@em0)" has resumed the state "MASTER" for vhid 2
Oct  4 09:53:25 OPN02 kernel: carp: 3@em2: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:53:25 OPN02 kernel: ifa_maintain_loopback_route: deletion failed for interface em2: 3
Oct  4 09:53:25 OPN02 kernel: carp: 2@em0: MASTER -> BACKUP (more frequent advertisement received)
Oct  4 09:53:25 OPN02 kernel: ifa_maintain_loopback_route: deletion failed for interface em0: 3
Oct  4 09:53:26 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.100.254 - DMZ-CARP (3@em2)" has resumed the state "BACKUP" for vhid 3
Oct  4 09:53:26 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "172.21.3.65 - WAN-CARP-01 (2@em0)" has resumed the state "BACKUP" for vhid 2
Oct  4 09:53:27 OPN02 kernel: carp: 1@em1: BACKUP -> MASTER (master timed out)
Oct  4 09:53:27 OPN02 opnsense: /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member "192.168.56.254 - LAN-CARP (1@em1)" has resumed the state "MASTER" for vhid 1


Obviously I switched interface assignment between WAN and LAN, but no difference. Thus it does not seem to be hardware related.

Analyzing the logs, I noticed that this line can put us on the right path to understand:

OPN01 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for WAN(wan) but ignoring since interface is configured with static IP (172.21.3.66 :: )

It's shown also when DMS interface goes down:

OPN01 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for DMZ(opt2) but ignoring since interface is configured with static IP (192.168.100.66 :: )

But when the LAN goes down (it has too its IPv4 address statically configured), here it is:

OPN01 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for lan

And this seems to cause vhid to be deleted from LAN interface, which I guess causes the inability to preempt other vhid's:

OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em1: 3
OPN01 kernel: ifa_maintain_loopback_route: deletion failed for interface em1: 3
OPN01 kernel: carp: demoted by -240 to 0 (vhid removed)
OPN01 kernel: em1: promiscuous mode disabled


Thanks for your help.

Sorry, but I really need help to understand OPNsense's behavior in this scenario. I'm experiencing the same issue described by @beloc, but only when LAN interface goes down (preemption just works with WAN link down).

With the hope to be able to migrate all our production pfSense clusters, I started testing OPNsense 18.7.4, but in the same simple scenario (identical hardware on both nodes, WAN, LAN and SYNC dedicated interfaces, CARP on both WAN and LAN), OPNsense behaves differently from pfSense, which always preempt when both LAN or WAN interface goes down.

I even tried the same setup with vanilla FreeBSD 11.1, and it behaves exactly like pfSense... thus, even if I was hoping it was my fault, I'm really starting thinking there is something wrong in this release of OPNsense.

I tested with the default settings: net.inet.carp.preempt always set to "1" on both nodes, and only OPNsense fails to preempt WAN's CARP address when LAN links goes down. Obviously, if I disable it on primary node it works, but in that way CARP's addresses never fails back to primary node when it comes back online.

I didn't find anything which could explain this behavior in forum and documentation.

Thanks for any help you might give me!