OPNsense Forum

Archive => 17.1 Legacy Series => Topic started by: Taomyn on March 29, 2017, 02:23:43 pm

Title: Not rebooting properly during console upgrade
Post by: Taomyn on March 29, 2017, 02:23:43 pm
Thought I would report this as it's now the 3rd time in a row where upgrading the firewall from the console seems to get stuck during the reboot.


I've waited well over 10 minutes, but always end up having to press Ctrl-C, then choosing "Reboot" from the menu.


Nothing seems to be harmed afterwards, I just never had this with v16.
Title: Re: Not rebooting properly during console upgrade
Post by: franco on March 29, 2017, 02:43:15 pm
One of the services is stuck, if it's not already exited. What's in /etc/rc.conf and /etc/rc.conf.d?
Title: Re: Not rebooting properly during console upgrade
Post by: Taomyn on March 29, 2017, 02:59:46 pm
Code: [Select]
root@bart:~ # ls /etc/rc.conf.d
acme_http_challenge     flowd_aggregate         netflow
captiveportal           haproxy                 squid
flowd                   ipfw                    suricata
root@bart:~ # ls /etc/rc.conf
ls: /etc/rc.conf: No such file or directory
Title: Re: Not rebooting properly during console upgrade
Post by: franco on March 29, 2017, 03:01:50 pm
Ok, need the internals... cat /etc/rc.conf.d/*

(make sure to scrub output if there is something private in there)
Title: Re: Not rebooting properly during console upgrade
Post by: Taomyn on March 29, 2017, 03:09:55 pm
Code: [Select]
acme_http_challenge_enable=YES
acme_http_challenge_conf="/var/etc/lighttpd-acme-challenge.conf"
acme_http_challenge_pidfile="/var/run/lighttpd-acme-challenge.pid"
acme_http_challenge_opnsense_bootup_run="/usr/local/opnsense/scripts/OPNsense/AcmeClient/setup.sh"
captiveportal_enable="NO"
#
# Automatic generated configuration for netflow.
# Do not edit this file manually.
#
flowd_enable="YES"
#
# Automatic generated configuration for netflow.
# Do not edit this file manually.
#
flowd_aggregate_enable="YES"
haproxy_enable=YES
haproxy_opnsense_bootup_run="/usr/local/opnsense/scripts/OPNsense/HAProxy/setup.sh"
haproxy_pidfile="/var/run/haproxy.pid"
haproxy_config="/usr/local/etc/haproxy.conf"
# haproxy_flags=""
firewall_enable="NO"
firewall_script="/usr/local/etc/rc.ipfw"
dummynet_enable="YES"
#
# Automatic generated configuration for netflow.
# Do not edit this file manually.
#
netflow_enable="YES"


0'
.?1'
..
0'squid¦¦suricata_enable="YES"
suricata_opnsense_bootup_run="/usr/local/opnsense/scripts/suricata/setup.sh"


# IPS mode, switch to netmapsuricata_netmap=YES
Title: Re: Not rebooting properly during console upgrade
Post by: franco on March 29, 2017, 03:42:23 pm
It could be HAproxy blocking?

Squid has a directory, I forgot...

# cat /etc/rc.conf.d/squid/*
Title: Re: Not rebooting properly during console upgrade
Post by: Taomyn on March 29, 2017, 03:46:30 pm
Code: [Select]
root@bart:~ # cat /etc/rc.conf.d/squid/*
squid_enable=NO

Would I be right that the last PID mentioned in the screen shot is the one for the process it's waiting for? If so, I'll try to remember if it happens next update to look up the process from it.
Title: Re: Not rebooting properly during console upgrade
Post by: franco on March 29, 2017, 04:57:12 pm
I think the addition of hooking into /etc/rc.shutdown caused this, added in 17.1-RC1:

https://github.com/opnsense/changelog/blob/922038/doc/17.1/17.1.r1#L43

SSH should still run, good idea to look for the pid. Theoretically, however, it isn't there anymore and it waits in vain. In that case, find the /var/run/*.pid file that has the actual PID to reveal the service name.


Thanks,
Franco
Title: Re: Not rebooting properly during console upgrade
Post by: Taomyn on March 29, 2017, 05:05:16 pm
Ok, would that also explain why earlier I tried twice to reboot from the console menu, and only when I went to the shell and typed "reboot" did it actually reboot?

I'm was not able to see the main screen at the time to see if it showed anything, but the SSH session did not show any PIDs it was waiting for.
Title: Re: Not rebooting properly during console upgrade
Post by: akron on June 04, 2017, 05:59:40 pm
Hi,

I am experiencing something similar, have an HA Setup with HA proxy and the master Firewall does not reboot or power off without going to the console and entering reboot or poweroff.

Backup Firewall reboots without problems when is not being used but If I failover the master one to the backup firewall, backup firewall does not reboot with the same behaviour.

Could this be a bug in HA config or similar?

Cheers
 
Title: Re: Not rebooting properly during console upgrade
Post by: akron on June 08, 2017, 02:29:39 pm
Ok, would that also explain why earlier I tried twice to reboot from the console menu, and only when I went to the shell and typed "reboot" did it actually reboot?

I'm was not able to see the main screen at the time to see if it showed anything, but the SSH session did not show any PIDs it was waiting for.

Did you discover the PID causing this issue?

I am still facing the problem I suspect HAproxy is the one, in theory should fail-over without problem to the Backup FW

Cheers
Title: Re: Not rebooting properly during console upgrade
Post by: Taomyn on June 08, 2017, 05:16:07 pm
Going by my reply in another thread, yes haproxy was the culprit:


I upgraded from 17.1.6-amd64 to 17.1.7 via the console option the following happened:


1. The upgrade could not reboot as it was waiting for a process, which when I killed simply killed my external connection. The process was "haproxy". When I arrived home I was unable to ssh to the box, my password was refused, used the console directly, root/no password and issued  a "reboot".
Title: Re: Not rebooting properly during console upgrade
Post by: akron on June 09, 2017, 10:06:34 am
Going by my reply in another thread, yes haproxy was the culprit:


I upgraded from 17.1.6-amd64 to 17.1.7 via the console option the following happened:


1. The upgrade could not reboot as it was waiting for a process, which when I killed simply killed my external connection. The process was "haproxy". When I arrived home I was unable to ssh to the box, my password was refused, used the console directly, root/no password and issued  a "reboot".

Ok cool, did you found any work around for this?

Because HA works well if I cut the network immediately fails-over to the backup firewall, but if I restart the main Firewall hangs over and does not failover causing interruption of service.

Cheers