Messages

That solved the problem! It was a network group Alias that contained mixed entries (ports and hosts).

Thank you very much! :)

Even though vmxnet3 is fully supported by OPNsense, the problem was solved by switching to Intel E1000E interfaces. There are no more VPN problems and even the bandwidth is slighly higher.

Hi,

we have a problem on our firewall cluster. Newly created firewall rules are not being used and there log entries in the "General Log" like this:

Code Select Expand

/usr/local/etc/rc.filter_configure: The command '/sbin/pfctl -f /tmp/rules.debug.old' returned exit code '1', the output was '/tmp/rules.debug.old:1321: syntax error /tmp/rules.debug.old:1348: syntax error /tmp/rules.debug.old:1349: syntax error pfctl: Syntax error in config file: pf rules not loaded'
If we look at /tmp/rules.debug.old at line 1321 we can't see any obvious errors:

Code Select Expand

   1312 # [prio: 400000]
   1313 pass in quick on vlan0.1.10 reply-to ( vlan0.1.10 10.10.0.254 ) inet proto carp from {any} to {any} keep state label "a3d5cd480d3e1f13588aa9e2dbcaff2f"
   1314 pass in quick on igc0 inet proto carp from {any} to {any} keep state label "3ef8dad901f0cc3ac0ba866b418b684d"
   1315 pass in quick on igc0 inet from {(igc0:network)} to {any} label "0b33b56fdd97fae9a15f40fce9f3730d" # Default allow LAN to any rule
   1316 pass in quick on igc0 inet6 from {(igc0:network),fe80::/10} to {any} label "4b503fa62fb7fff5813e3eb77a9971de" # Default allow LAN IPv6 to any rule
   1317 pass in quick on igc0 inet proto tcp from $ZABBIX_AGENTS to $ZABBIX_SERVER port {10051} keep state label "516f34385afd5e940fbb90604a6763a3" # Zabbix Agent to Server
   1318 pass in quick on igc0 inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "70eb3177ad6ec39449e5adbbe6a44c97" # Zabbix server to agent (TEMP)
   1319 pass in quick on igc0 inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "43facf5c455a797c7f2bab8f62cbfbd2" # Zabbix server to SNMP agent (get) (TEMP)
   1320 pass in quick on igc0 inet proto udp from $SNMP_AGENTS to $ZABBIX_SERVER port {162} keep state label "762f74ff93bc97d3c241a84c8c873056" # SNMP agent to Zabbix server (traps)
   1321 pass in quick on igc0 inet proto udp from $LOG_AGENTS to $LOG_UDP port {5010} keep state label "92423e0a66d994b02caf01f83d3f75e3" # SNMP agent to Zabbix server (traps)
   1322 pass in quick on igc0 inet proto udp from $SWITCH to $AD_DNS port {53} keep state label "b959a804d42a3c1c672704038db9eb61" # DNS AD
   1323 pass in quick on igc0 inet proto udp from $SWITCH to $AD_NTP port {123} keep state label "5d9710e886c2ac88a0f10d1e74875479" # NTP AD
   1324 pass in quick on igc1 inet from {any} to {any} keep state label "92a402b9c785d25842de702a547ee042"
   1325 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet proto carp from {any} to {any} keep state label "00e3a29ec800880646e46c0d57583e83"
   1326 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet proto tcp from {(vlan0.1.25:network)} to $HTTP_HTTPS_SERVER_ADMIN port $HTTP_HTTPS keep state label "e9d7b8caaf0de33e1b73b4b172225293"
   1327 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet from {any} to {any} keep state label "c4df453e9712f4b49cd6f00e62a15348"
   1328 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "76e8a0056ab343713c418b031de04480" # Zabbix server to SNMP agent (get) (TEMP)
   1329 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto tcp from $USV to $HAPROXY_SERVER port {25} keep state label "9b4ebb4501c74e6d28de9eb7ec6322c4" # Zabbix server to SNMP agent (get) (TEMP)
   1330 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $SNMP_AGENTS to $ZABBIX_SERVER port {162} keep state label "ae55bb61932442fd7e9a67368b7159d1" # SNMP agent to Zabbix server (traps)
   1331 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $USV to $AD_DNS port {53} keep state label "f08e059a1c9529ee31daa0bdb5a0d7dd" # DNS AD
   1332 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $USV to $AD_NTP port {123} keep state label "e17f2f754f7bd8150e1bbd891ec2586c" # NTP AD
   1333 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto carp from {any} to {any} keep state label "df501ae9fc80fdd6bb7853188c343878"
   1334 #debug:Interface opt4 not found
   1335 # pass in quick on ##opt4## inet proto icmp from {any} to {any} keep state label "2cd32dbf69fdb3f9bb35b58d4ecca4d7" # ICMP
   1336 pass in quick on vlan0.1.4 inet proto carp from {any} to {any} keep state label "a87911524e60387211920529db6e703e"
   1337 pass in quick on vlan0.1.4 inet proto udp from {(vlan0.1.4:network)} to $EXT_DNSCOLT_SERVER port {53} keep state label "54c919ed6f516e1b0190fc5cf7f84025" # DNS Colt
   1338 pass in quick on vlan0.1.4 inet proto {tcp udp} from {(vlan0.1.4:network)} to !$RFC1918 port $HTTP_HTTPS keep state label "0fd6158f30db1ebcc099815bb6f19426" # ONLY INTERNET
   1339 pass in quick on vlan0.1.16 reply-to ( vlan0.1.16 10.16.0.254 ) inet proto carp from {any} to {any} keep state label "f841b270ad41a89a6bb9d2a0aad448ee"
   1340 pass in quick on vlan0.1.28 reply-to ( vlan0.1.28 10.128.6.254 ) inet proto carp from {any} to {any} keep state label "e334fe5c3ee5fd311a232d82df4e0d8b"
   1341 pass in quick on vlan0.1.54 reply-to ( vlan0.1.54 10.54.0.254 ) inet proto carp from {any} to {any} keep state label "f3148a64695b613aef1e7c66a9258e13"
   1342 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto carp from {any} to {any} keep state label "2782037d45b9f6cab2ceb1ad861eee97"
   1343 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "3df93a3b7d994951ff1680760150abfd" # Zabbix server to SNMP agent (get)
   1344 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "8ec0ac1305733a8ab130ce105a57edf7" # Zabbix server to agent
   1345 pass in quick on vlan0.1.26 reply-to ( vlan0.1.26 10.26.0.254 ) inet proto carp from {any} to {any} keep state label "45ab7011cc5c1d23203eb299ede3ce6a"
   1346 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "99ab6978fd43795a32de7c9d94b219e2" # Zabbix server to agent (TEMP)
   1347 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto tcp from {(vlan0.1.39:network)} to $ZABBIX_SERVER port {10051} keep state label "31b16601dcbfb81392fdf07722d596e4" # Zabbix Agent to Server
   1348 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto udp from $LOG_UDP to $AD_NTP port {123} keep state label "ff31934ddfddbe0dd87b856aeb269309" # NTP AD
   1349 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto udp from $LOG_UDP to $AD_DNS port {53} keep state label "edc4ec2e63a31d90b1bbc699b5fda83f" # DNS AD
   1350 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto carp from {any} to {any} keep state label "2e4943600fb08621884b1bf367a0aaf1"
   1351 pass in quick on vlan0.1.70 inet proto udp from {(vlan0.1.70:network)} to $AD_SD_SERVER keep state label "4fe82b0a0b9997006ef4b698c962a77a" # PXE BOOT - TODO HIGH PORT RANGE
   1352 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_SD_SERVER port $AD_SD_TCP keep state label "0c46f9da349951bb40829d453cb69ff6"
   1353 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $HTTP_HTTPS_SERVER port $HTTP_HTTPS keep state label "527a3aa71919d7dba7f2c0b71c244536"
   1354 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $INTERNETPROXY_SERVER port $INTERNETPROXY_TCP keep state label "29d30643127f310a1e76d7b45caf91fb"
   1355 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_PRINT_SERVER port $AD_SMB_TCP keep state label "ccddd15b9fa0898a71130792ce761c68" # UNTESTED
   1356 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_R_SERVER port $AD_R_TCP keep state label "8b3d105b96f05205988327acc8045a06"
   1357 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_MAXQDA_SERVER port $AD_MAXQDA_TCP keep state label "8518bfb421d86bbceb95e50fcbaaeffd"
The corresponding rule is not visible in the OPNsense GUI and is also not present in the backup XML. The problem also persist after a reboot.

Is there a way to resolve this?

Yes, I installed os-vmware on OPNsense. The interface type is vmxnet3.

Hi all,

we have strange issues with some OPNsense VMs that run on VMware. The firewalls have a bunch of VPN site-to-site tunnels (IPsec and OpenVPN) which are mostly used for RDP sessions.
Every few days up to two weeks, all RDP sessions drop an can not be established again on all tunnels. If we ping inside the tunnel we have a high package loss (>50%) and the round trip times are all over the place. If we restart the tunnels, the problems persist. If we restart the whole firewall, the problems are gone.

We use the vmxnet3 adapters in VMware and the os-vmware plugin, if that matters.

If someone has or had similiar issues, I would be grateful for any help or hints! :)

Thank you very much. That worked!

I still used the legacy GUI for the IPsec tunnel (shame on me), but I reconfigured it in the new interface with your settings and now it works.

Thank you! :)

Thank you very much for your input!

I have set up a test connection (location B to location C) with an IPsec site2site tunnel to see what is going on the other side (C). The SPD entry worked and the traffic was routed through the tunnel, although the traffic never reached the LAN interface on location C. So pinging the firewall C didn't work, although the firewall rules would allow it.

When I add a normal Phase 2 entry for the OpenVPN net it works though. Am I missing something with SPD entry?

Thank you for your reply!

It isn't at the moment. My idea was to use outbound NAT for the OpenVPN net with the LAN IP to bypass this, because I have no access to the firewall on location A. Is this a bad idea or even possible?

Hi all,

I have a problemwith a firewall setup and don't know exactly how to solve this.

There is a OPNsense firewall (B) doing client VPN via OpenVPN. The firewall (B) also has a IPsec site2site tunnel to a different location (A). The problem is, that the traffic coming from the OpenVPN net is not routed over the site2site tunnel if the target is in the remote location (A).

[Location A] <----- IPsec site2site -----> [Location B OPNsense] <----- OpenVPN clients

Location A:
- 192.168.50.0/24

Location B:
- 192.168.248.0/24

OpenVPN net:
- 10.200.13.0/24
- pushed routes 192.168.50.0/24,192.168.248.0/24

If I ping a host on location B from the OpenVPN client it works. If I ping a host on location A the packet is directly routed over the WAN interface of the OPNsense and never enters the IPsec tunnel.

Can someone help me identifying this problem?

Thank you for the registry keys! We just started to investigate the problems, but this could be very helpful going forward.

[-f]

Thank you for your reply!

I forgot to mention that I used the standard Windows ping for my tests. So -f should be Don't fragment und -l should be size.

Is there any other method I can test if just setting the MSS on the IPsec interface is sufficient?

Hi all,

I have performance problems with RDP with some of our virtualized OPNsenses. These firewalls are hosted in a virtual environment and the clients are all connected via IPsec Site2Site tunnels.

[Client]----[Firewall]--------IPsec--------[OPNsense]----[RDP-Server]

The hoster suggested to set the MSS to 1300 for IPsec connections. Which I did in Firewall -> Settings -> Normalization -> Max mss 1300 for the IPsec interface. To test if this setting works, I tried to ping over the tunnel with a payload bigger than 1300 and the "Don't Fragment" flag.

Code Select Expand

ping -f -l 1472 4.3.2.1

I can ping with a size up to 1472 over the tunnel, which should not be possible right? Or do I have to set this on the LAN interface, too? I'm also puzzled how this is possible at all, if the hoster says that 1300 is their max.

[Source/Invert]

We do it like this:

• Create an Alias that contains all private networks as defined in RFC1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16).
• Create a firewall rule that only allows traffic to that Alias and use the Source/Invert checkbox. This way you'll only allow traffic not directed to private networks
• Optional: Create firewall rules to allow traffic that is directed to the firewall (i.e. DNS) and place it before the RFC1918 rule. This way you can make exceptions for traffic that should reach some of your private network destinations

No idea anybody?

Hi all,

I have a problem understanding as how to set this scenario up properly and need some help.

We have an OPNsense Firewall in a hosted cloud environment. We ordered 8 IPv4 addresses for our WAN interfaces and the provider delivers these as individual interfaces. So every IP is bound to a specific interface. On top of that, these IPs are in two different subnets with individual gateways.

So it's something like this:

• WAN_1: 1.0.0.1/24
• WAN_2: 1.0.0.2/24
• WAN_3: 1.0.0.3/24
• WAN_4: 1.0.0.4/24
• WAN_5: 2.0.0.1/24
• WAN_6: 2.0.0.2/24
• WAN_7: 2.0.0.3/24
• WAN_7: 2.0.0.4/24

I have configured two gateways, 1.0.0.254 and 2.0.0.254, where the first is the active gateway.

So my question is: How can I select which WAN interface uses which gateway? In the interface settings I can choose the gateway for one interface, but for the rest I can just select "Auto-detect". If I try to create a gateway for each WAN interface, it tells me that a gateway with this IP already exists.