Messages

I am also on a fresh install.

Seems like creating /etc/ssl/blacklisted fixes the issue.

[The only way I get this to work is when I add a static route via one of the remote gateways in the transport networks. Adding both doesn't really help in the case of fail-over as there's always just a single route in the routing table.]

I have successfully setup a gateway group to for two remote VPN gateways (remote appliance has two WAN links).

• Local LAN: 192.168.20.0/24
• Remote network: 172.16.0.0/16
• IPSec transport networks: 10.10.253.0/24, 10.10.254.0/24

I am directing traffic to the remote network via an incoming firewall rule on our internal interfaces:

Code Select Expand

Allow IPv4 - any protocol - from: anywhere - dst: remote network - gateway: gateway group

I added two incoming rules to the IPSec interface:

Code Select Expand

Allow IPv4 - any protocol - from: remote network - dst: anywhere
Allow IPv4 - any protocol - from: IPSec transport network - dst: anywhere

I can ping the remote site fine - the problem is, the remote site can't ping anything in our local network.
On the remote firewall I can ping the gateway interfaces fine.

I performed a packet capture and I see the following:

Code Select Expand

enc0 10:28:15.045875 (authentic,confidential): SPI 0xc96d654d: IP 172.16.1.199 > 192.168.20.29: ICMP echo request, id 1, seq 8474, length 40
ix0_vlan20 10:28:15.045901 IP 172.16.1.199 > 192.168.20.29: ICMP echo request, id 1, seq 8474, length 40
ix0_vlan20 10:28:15.046003 IP 192.168.20.29 > 172.16.1.199: ICMP echo reply, id 1, seq 8474, length 40


It looks like the ICMP echo reply is lost on its way back to the gateway group. Is there something I am missing?

The only way I get this to work is when I add a static route via one of the remote gateways in the transport networks. Adding both doesn't really help in the case of fail-over as there's always just a single route in the routing table.

This is driving me crazy for some time now - I am short of trying some dynamic routing protocols...

If for some reason you can't use dns-01 with LetsEncrypt, you can still make Opnsense procure the certificates (ACME plugin). The certs can then be copied to your internal services using auto-deploment rules in the plugin.

I have successfully setup a gateway group to for two remote VPN gateways.

• Local LAN: 192.168.20.0/24
• Remote network: 172.16.0.0/16
• IPSec transport networks: 10.10.253.0/24, 10.10.254.0/24

I am directing traffic to the remote network via an incoming firewall rule on our internal interfaces:

Code Select Expand

Allow IPv4 - any protocol - from: anywhere - dst: remote network - gateway: gateway group

I added two incoming rules to the IPSec interface:

Code Select Expand

Allow IPv4 - any protocol - from: remote network - dst: anywhere
Allow IPv4 - any protocol - from: IPSec transport network - dst: anywhere

I can ping the remote site fine - the problem is, the remote site can't ping anything in our local network.
On the remote firewall I can ping the gateway interfaces fine.

I performed a packet capture and I see the following:

Code Select Expand

enc0 10:28:15.045875 (authentic,confidential): SPI 0xc96d654d: IP 172.16.1.199 > 192.168.20.29: ICMP echo request, id 1, seq 8474, length 40
ix0_vlan20 10:28:15.045901 IP 172.16.1.199 > 192.168.20.29: ICMP echo request, id 1, seq 8474, length 40
ix0_vlan20 10:28:15.046003 IP 192.168.20.29 > 172.16.1.199: ICMP echo reply, id 1, seq 8474, length 40


It looks like the ICMP echo reply is lost on its way back to the gateway group. Is there something I am missing?

I believe, I went to the bottom of the issue.

(We are routing SMB over OPNsense which might not be a common use case.)

By default Suricata watches an SMB stream from beginning to end. In the case of long-lived SMB connections this might be a period of many days.
We assessed that our current signatures are more likely to trigger on connection setup. Therefore we limited the stream depth to 32mb and Suricata runs stable now. (https://forum.suricata.io/t/suricata-memory-allocation/573)

Code Select Expand

    smb:
      enabled: yes
      detection-ports:
        dp: 139, 445

      # Stream reassembly size for SMB streams. By default track it completely.
      # limited to avoid memory exhaustion
      stream-depth: 32mb

Any chance for this to be incorporated into Suricata's settings?

I am going to answer myself...

Looks like in FreeBSD it is not possible to add a second route to the same subnet:

Code Select Expand

fib 0: route already in table

Hence one has to use a proper routing protocol or use policy based routing.

I opted for the latter by performing these steps:

• create gateway group containing my two remote VPN gateways
• add a firewall rule on the interface receiving the incoming traffic (from my local LAN) destined for the remote network and choosing the gateway group as gateway
• remove the previously set static routes from config and from route status

The connectivity to the remote network was interrupted for a few minutes (I guess due to some firewall states), but eventually it worked.

I upgraded right away, thanks for the new release.

Turns out the culprit is a backup (Urbackup) which pulls approx. 60GB across the firewall (router between management and company network).
In some cases Suricata recovers bevore reaching 100% memory usage and then gets back to normal. So it's not a memory leak.

Question is why does Suricata build up such a large buffer?

We are already sending logs to a remote syslog (Fluentd/Opensearch).
Besides there were only 20+ messages that day.

The rule-update happens 16 hours before the system becomes unresponsive. We only use ET pro telemetry - somebody else should be faced with similar problems if the rules were broken.

From the Changelog in 22.1.9 I saw some memory leak in BSD13 being fixed. Will there be a new release for the business edition as well?

Thanks,
Alex

We're on 22.4.1 now and tested 22.1.x before.
At approx. 22.1.4 a strange behavior was introduced; our rock solid OPNsense started to hang, some traffic might pass but new VPN connections wouldn't, neither the web ui.

Unsure whether this might be a general memory leak we continued to monitor as to what the cause of the excessive memory consumption might be.
Turned out the no.1 memory eater was Suricata. If that's disabled everything is fine.

We have 16GB of RAM, about 2.7GB are generally used when Suricata doesn't run - about 3.7GB when it does.
There is a remote log to Fluentd/Opensearch setup via syslog. There are a few hundred events per day.
We run the ET Telemetry rules - most of them are enabled in IDS mode.

There is a Suricata restart Cron (I don't remember having set this up) at 01:2x in the morning.
About 20 hours later (at about 21:00) Monit starts sending Memory Limit exceeded warnings. Soon thereafter the box is not responding to anything anymore. There is no Cron entry at oraround that time.

Has anybody experienced something similar?
What else can we do to further debug the issue?

I created two routes to the same subnet using two different remote (VPN) gateways.
In Linux a distance parameter can be added for preferring one route over the other.

Is this possible as well or do I have to setup a Gateway group and try the counter-intuitive way via Firewall rules?

Or do I even have to setup a proper routing protocol?

I have my users logon with their login credentials to the firewall.
Access rights granted are the change password screen (where they can setup OTP) and the VPN config screen as I showed in the attached image.

As i said, currently this exposes all VPN configs and only works in the totally unsecure way if there's a single VPN config for everybody.

IMHO it should be doable to attach a GUI-ACL to each VPN config export. That way a user would only see the config which contains his certificate. If somebody could point me at where and how to implement, I'd be more than willing to :)

You could create a group and assign GUI privileges to members of that group.

Beware, that exposes all VPN configs!!!

I would welcome improvements about attaching ACLs to specific configurations :)

I renamed the topic as I am guessing this might have something to do with a virtual IP.

This is what I am trying to route:

Interface LAN: 192.168.20.1
Interface LAN: 192.168.30.1 (virtual IP)

Networks to be routed to the remote IPSec site:
192.168.20.0/24
192.168.30.0/24

Any ideas?

Pfelk bringt schon eine Aufbereitung der IP Adressen für GeoIP mit.
Für eigene Dashboards kannst du aber auch den Logstash GeoIP Filter nutzen.

I could not find any definition of the "OpenVPN net" either.

When running more than one OpenVPN servers (and networks) - does that network designate all OpenVPN networks?