Messages

- Is the OPNsense the default gateway for the Ubuntu VMs in that remote network?

No, it's not.

- Do the VMs have an active Firewall like ufw or iptables that could block the paket from source 172.16.23.0/24?

No, iptables -L returns an empty rule set.

Sure, with pleasure:

client <172.16.23.100> <-> local Os <172.16.23.1> <-> IPsec Tunnel/remote Os <10.2.2.5> <-> server <10.2.2.2>

Does this fit the purpose?

Thank you for your interest!

In a public cloud environment, an OPNsense 23.7.12_5 instance was installed as a simple IPsec gateway.

The OPT2 interface is connected to a 10.2.2.0/24 network with IP address 10.2.2.5/32, because autoconfigured ubuntu servers with similar /32 addresses reside there, e.g., 10.2.2.2/32:

server:

Code Select Expand

$ ip r | grep 10.2.2
10.2.2.0/24 via 10.2.2.1 dev enp7s0
10.2.2.0/24 via 10.2.2.1 dev enp7s0 proto dhcp src 10.2.2.2 metric 100
10.2.2.1 dev enp7s0 scope link
10.2.2.1 dev enp7s0 proto dhcp scope link src 10.2.2.2 metric 100


In order to support this setup, I figured to configure the OPNsense (hopefully) similarly:

Code Select Expand

$ netstat -nr4 | grep 10.2.2
10.2.2.0/24        10.2.2.1           UGS      vtnet3
10.2.2.1           link#4             UHS      vtnet3
10.2.2.5           link#4             UH          lo0


It took defining a custom IPv4 Upstream Gateway with 10.2.2.1 in the interface and a route from 10.2.2.0/24 to 10.2.2.1.

Using this setup, it is possible to ping 10.2.2.2 from 10.2.2.5 and vice versa directly.

In legacy tunnel settings, an IPsec phase 2 tunnel IPv4 was created with local network 10.2.2.0/24.

In IPsec firewall rules, IPv4 ICMP to 10.2.2.0/24 is allowed.

I can ping the OPNsense (10.2.2.5) via tunnel just fine, but none of the systems behind the tunnel.

Live view shows two green entries for attempts to ping 10.2.2.2 behind the tunnel, one from the IPsec inwards (with my custom label), and one on the interface outwards (let out anything from firewall host itself). A tcpdump doesn't show any ICMP packages arriving at 10.2.2.2.

The local OPNsense 23.7.12_5 IPsec endpoint has a phase 2 entry to remote subnet 10.2.2.0/24.

It looks like a routing problem between the IPsec and the physical interface.

Any ideas, what I'm missing here?

Thanks, Franco!

webgui is listening on LAN interface only, and this setting wasn't changed since early setup of OPNsense.

Is the (LAN) interface setup somehow racing with webgui start here?

Hmm, this is looking interesting (from /var/log/lighttpd/latest.log):

<27>1 2023-05-31T23:00:25+02:00 miller.lisa.loc lighttpd 65088 - [meta sequenceId="1"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1216) [note] graceful shutdown started
<27>1 2023-05-31T23:00:25+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="2"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1909) server started (lighttpd/1.4.71)
<27>1 2023-05-31T23:00:27+02:00 miller.lisa.loc lighttpd 65088 - [meta sequenceId="3"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.2308) server stopped by UID = 0 PID = 44602
<27>1 2023-05-31T23:00:27+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="4"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/gw_backend.c.281) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-1: No such file or directory
<27>1 2023-05-31T23:00:27+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="5"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/gw_backend.c.281) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-0: No such file or directory
<27>1 2023-05-31T23:00:27+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="6"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/gw_backend.c.1007) all handlers for /index.php? on .php are down.
<27>1 2023-05-31T23:00:30+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="7"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/gw_backend.c.358) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-1  0 /tmp/php-fastcgi.socket
<27>1 2023-05-31T23:00:30+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="8"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/gw_backend.c.358) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-0  0 /tmp/php-fastcgi.socket
<27>1 2023-05-31T23:02:59+02:00 miller.lisa.loc lighttpd 65001 - [meta sequenceId="1"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.2308) server stopped by UID = 0 PID = 24407
<27>1 2023-05-31T23:03:04+02:00 miller.lisa.loc lighttpd 3112 - [meta sequenceId="2"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1909) server started (lighttpd/1.4.71)


After the restart, the backend connection fails for some reason!

Just noticed, that a 23.1.9 update is available.

Applied it, and rebooted (to test, if the behaviour may have changed).

Unfortunately not. Proper webgui is available after manual restart of lighttpd.
While at it, isn't the webgui a service? Right now, I start the webgui with:

/usr/local/sbin/lighttpd -f /var/etc/lighty-webConfigurator.conf

Guess, it would be cleaner to restart the appropriate service, but nothing jumped into my eye from the list of
service -l.

The system is a pretty standard Intel(R) Celeron(R) N5105 with four I225 nics: (WAN, LAN, DMZ), running OPNsense 23.1.9-amd64.

[version info corrected!]

Hello,

after today's upgrade to 23.1.8 (not exactly sure, from which version, last update was around March), the login screen is displayed fine, but attempting to login results in a 503.

Connecting via ssh, I can see the lighttpd process running, and also checked /var/etc/lighty-webConfigurator.conf, that it picked up my non-standard https port. Killing and restarting the lighttpd process gives back the webgui.

The effect is reproducible after reboot: Reboot msg -> Login -> 503 -> kill -> manual restart -> webgui.
Everything else seems to be working fine so far.

I'm probably too new to OPNsense to further debug this issue without a helping hand.