Messages

1

23.1 Legacy Series / Re: IPv6 is broken after Update to 23.1.4 + 503 Service unavailable error

« on: March 25, 2023, 10:17:37 pm »

Looks like the persistence to get this minor issue fixed made the problem worse on 23.1.4. I've written a new patch that should sidestep the underlying issue:

https://github.com/opnsense/core/commit/33ad50456

# opnsense-patch 33ad50456

Please report back if it works or not...


Cheers,
Franco

I have the Web GUI set to run on two Listen Interfaces: Mgmt (management VLAN), OPT1 (personal network interface for my one PC)
System -> Settings -> Administration -> Web GUI -> Listen Interfaces

Reason: Lowers the attack surface area, the other VLANs and networks do not need nor should ever be accessing the WebGUI so if it is not listening at all it is better off.
the SSHd is also setup the same way.

OPT1 also has IPv6 enabled and that has been working great with the fix in 23.1.4_1. DHCPv6 and radvd error etc are gone.

With the above 33ad50456 patch applied I was able to reproduce the 503 Service Unavailable error on the subsequent reboot. Before I didn't know when it would happen just the next time I tried to open the WebGUI it would have the service unavailable error.
configctl webgui restart via SSH would get it to work again for awhile.

My hunch from what I am reading in this thread is the interface up/down is triggering the issue and it will likely happen after one or more restarts of my OPT1 connected PC.

I also do not see any PHP errors

Code: [Select]

cat /tmp/PHP_errors.log
cat: /tmp/PHP_errors.log: No such file or directory

Web GUI Log file and what happened between the entries

Code: [Select]


action: reboot after applying opnsense-patch 33ad50456
2023-03-25T14:20:06-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.1704) server started (lighttpd/1.4.69)

action: connection timeout after attempting to login to web gui after reboot.
action: "configctl webgui restart" issued via SSH
2023-03-25T14:20:23-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.1057) [note] graceful shutdown started
2023-03-25T14:22:12-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.1704) server started (lighttpd/1.4.69)

action: 503 Error from the web gui after login and just checking out the web gui log file. This was new as haven't been actively using the web gui when it threw the 503 error before.

2023-03-25T14:25:49-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/mod_openssl.c.3438) SSL (error): 5 -1: Operation timed out
2023-03-25T14:25:49-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.2078) server stopped by UID = 0 PID = 57711
2023-03-25T14:26:00-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/gw_backend.c.274) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-1: No such file or directory
2023-03-25T14:26:00-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/gw_backend.c.274) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-0: No such file or directory
2023-03-25T14:26:00-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/gw_backend.c.960) all handlers for /ui/index.php? on .php are down.
2023-03-25T14:26:03-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/gw_backend.c.351) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-1 0 /tmp/php-fastcgi.socket
2023-03-25T14:26:03-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/gw_backend.c.351) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-0 0 /tmp/php-fastcgi.socket

configctl webgui restart issued via SSH
2023-03-25T14:26:09-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.1057) [note] graceful shutdown started
2023-03-25T14:27:30-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.1704) server started (lighttpd/1.4.69)

while typing this up in the "General" log
No commands were issued to cause that error to my knowledge

Code: [Select]

2023-03-25T14:31:12-06:00 Error opnsense /usr/local/etc/rc.restart_webgui: The command '/usr/local/bin/flock -ne /var/run/lighty-webConfigurator.pid /usr/local/sbin/lighttpd -f /var/etc/lighty-webConfigurator.conf' returned exit code '1', the output was ''

Then in the Web GUI Log
2023-03-25T14:31:12-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/mod_openssl.c.3438) SSL (error): 5 -1: Operation timed out
2023-03-25T14:31:12-06:00 Error lighttpd (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.69/src/server.c.2078) server stopped by UID = 0 PID = 54531


The Web GUI is still working without 503 or connection timeout via the OPT1 interface...for now?
The current lighttpd pid according to top
87005 root          1  20    0    18M  7932K kqread   1   0:00   0.00% lighttpd

Hopefully this helps troubleshoot the issue.