Messages

[tagged]

Have you setup VLANs on your OPNsense already?

If you do use VLAN-Interfaces on your OPNsense: Set the ports on the switch that are connected to the OPNsense to tagged, as the switch now expects already VLAN tagged packets.
If you do not use VLAN-Interfaces on OPNsense: Set the ports on the switch that are connected to the OPNsense to untagged, as the switch now expects untagged packets and will tag them when they are processed.

Dear OPNsensers,

I have a weird problem with one of our OPNsenses and I can't find the reason for the behaviour.

The Webinterface is bound to port 4444 and reachable by a few public IPs on the WAN interface. My problem is, that when I want to access the GUI through one of these IPs, it works for a few minutes and then just breaks with a PR_END_OF_FILE_ERROR. I have to restart the firewall or reset the WAN interface (PPPoE over VLAN 7) for it to work again (for a few minutes).

Sometimes, but not always I can still do a

Code Select Expand

curl -v -k https://1.2.3.4:4444 and it works and sometimes not even SSH works anymore on the WAN interface. I also changed the certificate for the GUI, but no success. Firewall rules are all in place.

This is a curl when the connection does not work:

Code Select Expand

~# curl -v -k http://1.2.3.4:4444
*   Trying 1.2.3.4:4444...
* Connected to 1.2.3.4 (1.2.3.4) port 4444 (#0)
> GET / HTTP/1.1
> Host: 1.2.3.4:4444
> User-Agent: curl/7.88.1
> Accept: */*
>
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server
This is the analog problem with SSH.

Code Select Expand

ssh -v root@1.2.3.4
OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2
debug1: Reading configuration data C:\\Users\\my.user/.ssh/config
debug1: Connecting to 1.2.3.4 [1.2.3.4] port 22.
debug1: Connection established.
debug1: identity file C:\\Users\\my.user/.ssh/id_rsa type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_rsa-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ecdsa type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ecdsa-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ecdsa_sk type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ed25519 type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ed25519-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ed25519_sk type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_ed25519_sk-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_xmss type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_xmss-cert type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_dsa type -1
debug1: identity file C:\\Users\\my.user/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_9.5
kex_exchange_identification: Connection closed by remote host
Connection closed by 1.2.3.4 port 22
This is really confusing since I don't know where to even look at now.

We upgraded the firmware (BIOS) on one of our DEC3852 to the newest version. The appliance has 16 GB of RAM, but after the update only 8 GB are present. Is there something to do after the update? Unfortunately I have no access to the console at the moment, so it would be good to know if I missed something or if this is an coinsidence with a hardware defect.

That solved the problem! It was a network group Alias that contained mixed entries (ports and hosts).

Thank you very much! :)

Even though vmxnet3 is fully supported by OPNsense, the problem was solved by switching to Intel E1000E interfaces. There are no more VPN problems and even the bandwidth is slighly higher.

Hi,

we have a problem on our firewall cluster. Newly created firewall rules are not being used and there log entries in the "General Log" like this:

Code Select Expand

/usr/local/etc/rc.filter_configure: The command '/sbin/pfctl -f /tmp/rules.debug.old' returned exit code '1', the output was '/tmp/rules.debug.old:1321: syntax error /tmp/rules.debug.old:1348: syntax error /tmp/rules.debug.old:1349: syntax error pfctl: Syntax error in config file: pf rules not loaded'
If we look at /tmp/rules.debug.old at line 1321 we can't see any obvious errors:

Code Select Expand

   1312 # [prio: 400000]
   1313 pass in quick on vlan0.1.10 reply-to ( vlan0.1.10 10.10.0.254 ) inet proto carp from {any} to {any} keep state label "a3d5cd480d3e1f13588aa9e2dbcaff2f"
   1314 pass in quick on igc0 inet proto carp from {any} to {any} keep state label "3ef8dad901f0cc3ac0ba866b418b684d"
   1315 pass in quick on igc0 inet from {(igc0:network)} to {any} label "0b33b56fdd97fae9a15f40fce9f3730d" # Default allow LAN to any rule
   1316 pass in quick on igc0 inet6 from {(igc0:network),fe80::/10} to {any} label "4b503fa62fb7fff5813e3eb77a9971de" # Default allow LAN IPv6 to any rule
   1317 pass in quick on igc0 inet proto tcp from $ZABBIX_AGENTS to $ZABBIX_SERVER port {10051} keep state label "516f34385afd5e940fbb90604a6763a3" # Zabbix Agent to Server
   1318 pass in quick on igc0 inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "70eb3177ad6ec39449e5adbbe6a44c97" # Zabbix server to agent (TEMP)
   1319 pass in quick on igc0 inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "43facf5c455a797c7f2bab8f62cbfbd2" # Zabbix server to SNMP agent (get) (TEMP)
   1320 pass in quick on igc0 inet proto udp from $SNMP_AGENTS to $ZABBIX_SERVER port {162} keep state label "762f74ff93bc97d3c241a84c8c873056" # SNMP agent to Zabbix server (traps)
   1321 pass in quick on igc0 inet proto udp from $LOG_AGENTS to $LOG_UDP port {5010} keep state label "92423e0a66d994b02caf01f83d3f75e3" # SNMP agent to Zabbix server (traps)
   1322 pass in quick on igc0 inet proto udp from $SWITCH to $AD_DNS port {53} keep state label "b959a804d42a3c1c672704038db9eb61" # DNS AD
   1323 pass in quick on igc0 inet proto udp from $SWITCH to $AD_NTP port {123} keep state label "5d9710e886c2ac88a0f10d1e74875479" # NTP AD
   1324 pass in quick on igc1 inet from {any} to {any} keep state label "92a402b9c785d25842de702a547ee042"
   1325 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet proto carp from {any} to {any} keep state label "00e3a29ec800880646e46c0d57583e83"
   1326 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet proto tcp from {(vlan0.1.25:network)} to $HTTP_HTTPS_SERVER_ADMIN port $HTTP_HTTPS keep state label "e9d7b8caaf0de33e1b73b4b172225293"
   1327 pass in quick on vlan0.1.25 reply-to ( vlan0.1.25 10.25.0.253 ) inet from {any} to {any} keep state label "c4df453e9712f4b49cd6f00e62a15348"
   1328 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "76e8a0056ab343713c418b031de04480" # Zabbix server to SNMP agent (get) (TEMP)
   1329 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto tcp from $USV to $HAPROXY_SERVER port {25} keep state label "9b4ebb4501c74e6d28de9eb7ec6322c4" # Zabbix server to SNMP agent (get) (TEMP)
   1330 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $SNMP_AGENTS to $ZABBIX_SERVER port {162} keep state label "ae55bb61932442fd7e9a67368b7159d1" # SNMP agent to Zabbix server (traps)
   1331 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $USV to $AD_DNS port {53} keep state label "f08e059a1c9529ee31daa0bdb5a0d7dd" # DNS AD
   1332 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto udp from $USV to $AD_NTP port {123} keep state label "e17f2f754f7bd8150e1bbd891ec2586c" # NTP AD
   1333 pass in quick on vlan0.1.48 reply-to ( vlan0.1.48 10.48.0.254 ) inet proto carp from {any} to {any} keep state label "df501ae9fc80fdd6bb7853188c343878"
   1334 #debug:Interface opt4 not found
   1335 # pass in quick on ##opt4## inet proto icmp from {any} to {any} keep state label "2cd32dbf69fdb3f9bb35b58d4ecca4d7" # ICMP
   1336 pass in quick on vlan0.1.4 inet proto carp from {any} to {any} keep state label "a87911524e60387211920529db6e703e"
   1337 pass in quick on vlan0.1.4 inet proto udp from {(vlan0.1.4:network)} to $EXT_DNSCOLT_SERVER port {53} keep state label "54c919ed6f516e1b0190fc5cf7f84025" # DNS Colt
   1338 pass in quick on vlan0.1.4 inet proto {tcp udp} from {(vlan0.1.4:network)} to !$RFC1918 port $HTTP_HTTPS keep state label "0fd6158f30db1ebcc099815bb6f19426" # ONLY INTERNET
   1339 pass in quick on vlan0.1.16 reply-to ( vlan0.1.16 10.16.0.254 ) inet proto carp from {any} to {any} keep state label "f841b270ad41a89a6bb9d2a0aad448ee"
   1340 pass in quick on vlan0.1.28 reply-to ( vlan0.1.28 10.128.6.254 ) inet proto carp from {any} to {any} keep state label "e334fe5c3ee5fd311a232d82df4e0d8b"
   1341 pass in quick on vlan0.1.54 reply-to ( vlan0.1.54 10.54.0.254 ) inet proto carp from {any} to {any} keep state label "f3148a64695b613aef1e7c66a9258e13"
   1342 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto carp from {any} to {any} keep state label "2782037d45b9f6cab2ceb1ad861eee97"
   1343 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto udp from $ZABBIX_SERVER to $SNMP_AGENTS port {161} keep state label "3df93a3b7d994951ff1680760150abfd" # Zabbix server to SNMP agent (get)
   1344 pass in quick on vlan0.1.62 reply-to ( vlan0.1.62 10.62.0.254 ) inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "8ec0ac1305733a8ab130ce105a57edf7" # Zabbix server to agent
   1345 pass in quick on vlan0.1.26 reply-to ( vlan0.1.26 10.26.0.254 ) inet proto carp from {any} to {any} keep state label "45ab7011cc5c1d23203eb299ede3ce6a"
   1346 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto tcp from $ZABBIX_SERVER to $ZABBIX_AGENTS port {10050} keep state label "99ab6978fd43795a32de7c9d94b219e2" # Zabbix server to agent (TEMP)
   1347 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto tcp from {(vlan0.1.39:network)} to $ZABBIX_SERVER port {10051} keep state label "31b16601dcbfb81392fdf07722d596e4" # Zabbix Agent to Server
   1348 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto udp from $LOG_UDP to $AD_NTP port {123} keep state label "ff31934ddfddbe0dd87b856aeb269309" # NTP AD
   1349 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto udp from $LOG_UDP to $AD_DNS port {53} keep state label "edc4ec2e63a31d90b1bbc699b5fda83f" # DNS AD
   1350 pass in quick on vlan0.1.39 reply-to ( vlan0.1.39 10.39.0.254 ) inet proto carp from {any} to {any} keep state label "2e4943600fb08621884b1bf367a0aaf1"
   1351 pass in quick on vlan0.1.70 inet proto udp from {(vlan0.1.70:network)} to $AD_SD_SERVER keep state label "4fe82b0a0b9997006ef4b698c962a77a" # PXE BOOT - TODO HIGH PORT RANGE
   1352 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_SD_SERVER port $AD_SD_TCP keep state label "0c46f9da349951bb40829d453cb69ff6"
   1353 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $HTTP_HTTPS_SERVER port $HTTP_HTTPS keep state label "527a3aa71919d7dba7f2c0b71c244536"
   1354 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $INTERNETPROXY_SERVER port $INTERNETPROXY_TCP keep state label "29d30643127f310a1e76d7b45caf91fb"
   1355 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_PRINT_SERVER port $AD_SMB_TCP keep state label "ccddd15b9fa0898a71130792ce761c68" # UNTESTED
   1356 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_R_SERVER port $AD_R_TCP keep state label "8b3d105b96f05205988327acc8045a06"
   1357 pass in quick on vlan0.1.70 inet proto tcp from {(vlan0.1.70:network)} to $AD_MAXQDA_SERVER port $AD_MAXQDA_TCP keep state label "8518bfb421d86bbceb95e50fcbaaeffd"
The corresponding rule is not visible in the OPNsense GUI and is also not present in the backup XML. The problem also persist after a reboot.

Is there a way to resolve this?

Yes, I installed os-vmware on OPNsense. The interface type is vmxnet3.

Hi all,

we have strange issues with some OPNsense VMs that run on VMware. The firewalls have a bunch of VPN site-to-site tunnels (IPsec and OpenVPN) which are mostly used for RDP sessions.
Every few days up to two weeks, all RDP sessions drop an can not be established again on all tunnels. If we ping inside the tunnel we have a high package loss (>50%) and the round trip times are all over the place. If we restart the tunnels, the problems persist. If we restart the whole firewall, the problems are gone.

We use the vmxnet3 adapters in VMware and the os-vmware plugin, if that matters.

If someone has or had similiar issues, I would be grateful for any help or hints! :)

Thank you very much. That worked!

I still used the legacy GUI for the IPsec tunnel (shame on me), but I reconfigured it in the new interface with your settings and now it works.

Thank you! :)

Thank you very much for your input!

I have set up a test connection (location B to location C) with an IPsec site2site tunnel to see what is going on the other side (C). The SPD entry worked and the traffic was routed through the tunnel, although the traffic never reached the LAN interface on location C. So pinging the firewall C didn't work, although the firewall rules would allow it.

When I add a normal Phase 2 entry for the OpenVPN net it works though. Am I missing something with SPD entry?

Thank you for your reply!

It isn't at the moment. My idea was to use outbound NAT for the OpenVPN net with the LAN IP to bypass this, because I have no access to the firewall on location A. Is this a bad idea or even possible?

Hi all,

I have a problemwith a firewall setup and don't know exactly how to solve this.

There is a OPNsense firewall (B) doing client VPN via OpenVPN. The firewall (B) also has a IPsec site2site tunnel to a different location (A). The problem is, that the traffic coming from the OpenVPN net is not routed over the site2site tunnel if the target is in the remote location (A).

[Location A] <----- IPsec site2site -----> [Location B OPNsense] <----- OpenVPN clients

Location A:
- 192.168.50.0/24

Location B:
- 192.168.248.0/24

OpenVPN net:
- 10.200.13.0/24
- pushed routes 192.168.50.0/24,192.168.248.0/24

If I ping a host on location B from the OpenVPN client it works. If I ping a host on location A the packet is directly routed over the WAN interface of the OPNsense and never enters the IPsec tunnel.

Can someone help me identifying this problem?

Thank you for the registry keys! We just started to investigate the problems, but this could be very helpful going forward.

[-f]

Thank you for your reply!

I forgot to mention that I used the standard Windows ping for my tests. So -f should be Don't fragment und -l should be size.

Is there any other method I can test if just setting the MSS on the IPsec interface is sufficient?

Hi all,

I have performance problems with RDP with some of our virtualized OPNsenses. These firewalls are hosted in a virtual environment and the clients are all connected via IPsec Site2Site tunnels.

[Client]----[Firewall]--------IPsec--------[OPNsense]----[RDP-Server]

The hoster suggested to set the MSS to 1300 for IPsec connections. Which I did in Firewall -> Settings -> Normalization -> Max mss 1300 for the IPsec interface. To test if this setting works, I tried to ping over the tunnel with a payload bigger than 1300 and the "Don't Fragment" flag.

Code Select Expand

ping -f -l 1472 4.3.2.1

I can ping with a size up to 1472 over the tunnel, which should not be possible right? Or do I have to set this on the LAN interface, too? I'm also puzzled how this is possible at all, if the hoster says that 1300 is their max.