Messages

Hi

I've been running V24.1.9_3 since 20-Jun & everything appeared to be fine on the surface.  Today, since 24.1.10 became available, I cleaned my glasses properly & looked at the dashboard and realised that I had no IPV6 addresses.  (I do have a public V4 (and no CGNAT) which explains  why I (we) hadn't noticed that anything was awry before.

I rebooted the FW and noted that IPV6 was back ok.  I then set to, writing a logging script to check how long it took to fail.  My public V6 address had gone before I'd finished writing the script  ::) .   So I finished the script, rebooted again, and ran the logging script.  It took precisely 10 minutes to lose the V6 address again (sounds like a 600 sec lease to me).

I then did a backup of the config file, and tried to do the update to 24.1.10_1.  That didn't work.  I'm guessing that part of the system was trying to do the upgrade using IPV6, which had silently gone away.

So I rebooted again, and started the update to 24.1.10_1 before I lost my V6 public address.  The update appeared to go ok.  When it had finished, I did a manual reboot (but not a filter reload - because I wasn't sure what that was referring to) and the system has now been up for just over an hour.  My IPV6 logging script shows that my V6 address hasn't gone awol.

So 24.1.10_1 seems to have fixed whatever was wrong in 24.1.9_3 for me, at least.

I do have another live system, also running 24.1.9_3, but that system has a fixed IPV6 public address, but, unsurprisingly, that system shows absolutely no problem.

HTH

PeterF

Hi

I noticed that your traffic graph is displaying (only) Ipsec traffic.  If you want to look at WAN or LAN traffic, then you need to select those using the drop-down list (arrow to the right of "Traffic" on the tab).

Regards,

PeterF

[dyn_dns widget]

Hi

I encountered this problem earlier toay - with just the top & left side of the Webgui page visible.  I looked at the page source and at the end, there was a crash report indicting dyn_dns widget. 

Code Select Expand

Fatal error: Uncaught Error: Failed opening required 'plugins.inc.d/dyndns.inc' (include_path='/usr/local/etc/inc:/usr/local/www:/usr/local/opnsense/mvc:/usr/local/opnsense/contrib:/usr/local/share/pear:/usr/local/share') in /usr/local/www/widgets/widgets/dyn_dns_status.widget.php:34
Stack trace:
#0 /usr/local/www/index.php(413): include()
#1 {main}
  thrown in /usr/local/www/widgets/widgets/dyn_dns_status.widget.php on line 34


So (in a terminal window) I removed:

Code Select Expand

rm /usr/local/www/widgets/widgets/dyn_dns_status.widget.php

and

rm /usr/local/www/widgets/include/dyn_dns_status.inc
and then refreshed the page.  Having got the webgui back again, I was able to go & finish the cleanup in SYTEM : FIRMWARE...

HTH someone. 
Pity I hadn't found this page earlier - I could have prevented the problem  >:(

PeterF

Hi

Apologies for the delay (I've only just seen your post)...
I have been successfully using Toob (900Mb up & down - fibre) since December 2022.  It's my only wan connection from my OPNsense firewall.
IPV4 configuration was easy (DHCP).  The biggest challenge was finding out what Toob expected for IPV6.  Their help desk was completely unhelpful, so it was a matter of deduction from the configuration of their supplied router.  I got there in the end.
One gets a /64 (via DHCPv6) for the wan connection, and a /56 for one own consumption.

PeterF

Ok...got a fix, but no idea why/how it broke in the first place.

Glad it's sorted for you.  I'm going to watch my certificate the next time LE does an update and see what happens.

I think the protocol now is for you to insert a [SOLVED] at the beginning of your post title, if you're happy that it is.

Good luck!

PeterF

[old]

Hi DenverTech

I had a similar situation in the middle of March (2023) - I was running V23.1_6 at the time.  Whilst running V23.1_6 my Acme.sh had run successfully because a certificate update was needed.  I didn't bother looking at the certificate details, until I noticed that my browsers (I tried different browsers on different machines too) were all telling me that there was a certificate problem.  I looked at the certificate and sure enough it had expired.  I spent a few hours digging to try and understand how certificates are stored and referenced in OPNsense.  What follows is a summary of how certificates are stored, what caused the problem in my system, and how I fixed it.

All certiificates are stored in the config file ( /conf/config.xml ) in a structure that looks like the box below.  "<cert>" is at level 2 (with "<opnsense>" at level 1 (top)).  There is a separate "<cert>" section for each certificate.  The "<refid>" item is unique for every certificate, and is used to select a required certificate.

Code Select Expand


<?xml version="1.0"?>
<opnsense>
  <version>11.2</version>
  .
  <cert>
    <refid>a1b2c3d4e5f6</refid>
    <descr>Text Description</desc>
    <crt>[Very long continuous string of the public key of this certificate]</crt>
    <prv>[Very long continuous string of the private key of this certificate]</prv>
  </cert>


Now in another level 2 section denoted by "<system>", there exists a level 3 section denoted by "<webgui>".  See the next box for its structure:

Code Select Expand


<?xml version="1.0"?>
<opnsense>
  <version>11.2</version>
  .
  <cert>
    .
  </cert>
  .
  <system>
    .
    <webgui>
      <protocol>https</protocol>
      <ssl-certref>6045008dd0e08</ssl-certref>
      <port>8443</port>
      <ssl-ciphers/>
      <interfaces/>
      <compression>5</compression>
      .
      .
    <webgui>
    .
  </system>




The item "<ssl-certref>" contains the 12-digit identifier for the certificate that is to be used for the web server. 

In my system which was still serving the out-of-date certificate, the identifier (pointer) contained in the "<ssl-certref>" parameter was actually the id for the old certificate, not the new certificate that had been acquired by acme.sh.  That explained why the certificate being served by the web server, was out of date.

Fixing the problem

• I made a backup copy of the config.xml file
• I located the new acme.sh acquired certificate in its <cert>..</cert> block and noted its refid
• I edited the config file and updated the certificate reference in the "<ssl-certref>" section with the correct id from step 2
• Rebooted the system

That fixed the problem for me.  What I didn't do was locate the code that updates the <webgui> section after acme.sh has run and try to come up with an hypothesis as to why it wasn't updated correctly when acme.sh ran successfully.
I'll keep an eye on it the next time acme.ssh runs to see if it happens again.

An Aside

Having fixed the certificate pointer, I went and looked at the certificates from the GUI (System -> Trust -> Certificates ).
I then noticed that the entry for the old (out-of-date) certificate for (ACME Client) now had a little waste bin icon at the end of the line, indicating that it could be deleted.  When I had started my investigation I had noted that there was no waste bin icon for the old certificate.

HTH with your problem

PeterF

Hi

Have you checked the dates on both ends?
BTW, I use FreeOTP (Ver 2.0.1 (42)) on Android - works well & appears stable.

PeterF

[Then]

Hi all,

I am in the process of activating 2FA (TOTP) for all services that offer it. E.g. Github, our self hosted Gitlab, Hetzner, Paypal, ... you name it.

For all of these services the login procedure is the same:

1. Prompt for username and password. Sometimes first only username, <ENTER>, then password <ENTER>.
2. Then I am asked for the 6 digit one time token.

This works great because I have been using password safe software for years and the username and password get filled in automatically. Then I have one more step to enter the OTP. Perfect.

With OPNsense it seems there is only one prompt for username and password and you are supposed to append (or prepend depending on configuration) your OTP to the static password.

How is this expected to work? The password is filled in by the password manager. Visible as a bunch of dots or stars. If I append the OTP in that same field my browser asks me if I want to update the saved password for that service every single time.

WTF? What is the idea behind this completely insane user interface? Unless I get one prompt for fixed username and password and then a second one for the OTP, 2FA on OPNsense is unusable. Every other service I have ever used with TOTP does this.

Kind regards,
Patrick

I concur with Patrick on this one - having to remember the password (because I can't use one from my password manager) and type it, and the OTP into the PW field is an absolute pain. 

Please could we have the separate OTP field as a New Year present  ;)

Thanks for all the hard word.

Seasons greetings to all

PeterF

That sounds like a plan. I will try that tonight when I get home and report back...

I would also be inclined to create another file in that directory (/usr/local/etc I believe) with your own file name and contents.  Do this before rebooting, and check that it survives the boot process.  This would help to eliminate a file system etc problem/feature - because I don't think we know anything about your test system.

Just a thought...

PeterF

This seems to do the trick :D

https://github.com/opnsense/core/commit/2a3f201d38

It does indeed!  It took me a little while to figure it out until I read the man page for pkg.conf.

Many thanks for the help.  Can we mark this as closed now?

PeterF

Hi Franco

If that's the case I really wasn't expecting FreeBSD package manager would forget its good data due to not reaching its mirror via IPv6?

I've done a bit of digging to try & understand what is going on.
1  The CONNECTIVITY audit is implemented by this script:

Code Select Expand

cat /usr/local/opnsense/scripts/firmware/connection.sh

#!/bin/sh

# Copyright (C) 2021 Franco Fichtner <franco@opnsense.org>
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
#    this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
# OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

LOCKFILE="/tmp/pkg_upgrade.progress"
TEE="/usr/bin/tee -a"

: > ${LOCKFILE}

URL=$(opnsense-update -M)
POPT="-c4 -s1500"

HOST=${URL#*://}
HOST=${HOST%%/*}
IPV4=$(host -t A ${HOST} | head -n 1 | cut -d\  -f4)
IPV6=$(host -t AAAA ${HOST} | head -n 1 | cut -d\  -f5)

echo "***GOT REQUEST TO AUDIT CONNECTIVITY***" >> ${LOCKFILE}
echo "Currently running $(opnsense-version) at $(date)" >> ${LOCKFILE}
if [ -n "${IPV4}" -a -z "${IPV4%%*.*}" ]; then
        echo "Checking connectivity for host: ${HOST} -> ${IPV4}" | ${TEE} ${LOCKFILE}
        (ping ${POPT} ${IPV4} 2>&1) | ${TEE} ${LOCKFILE}
        echo "Checking connectivity for repository (IPv4): ${URL}" | ${TEE} ${LOCKFILE}
        (pkg -4 update -f 2>&1) | ${TEE} ${LOCKFILE}
else
        echo "No IPv4 address could be found for host: ${HOST}" | ${TEE} ${LOCKFILE}
fi
if [ -n "${IPV6}" -a -z "${IPV6%%*:*}" ]; then
        echo "Checking connectivity for host: ${HOST} -> ${IPV6}" | ${TEE} ${LOCKFILE}
        (ping6 ${POPT} ${IPV6} 2>&1) | ${TEE} ${LOCKFILE}
        echo "Checking connectivity for repository (IPv6): ${URL}" | ${TEE} ${LOCKFILE}
        (pkg -6 update -f 2>&1) | ${TEE} ${LOCKFILE}
else
        echo "No IPv6 address could be found for host: ${HOST}" | ${TEE} ${LOCKFILE}
fi
echo '***DONE***' >> ${LOCKFILE}

2  After the script has executed "pkg -4 update -f" (in my IPV4-only environment) in the IPV4 section, there exists an SQLite 3.x database file:

Code Select Expand

/var/db/pkg/repo-OPNsense.sqlite

3  When the second (IPV6) half runs (executing"pkg -6 update -f"), the database file has been deleted if there is no IPV6 connectivity.  The rest of the files in the same directory remain untouched.

Perhaps it would be better if this script only executed the pkg command if there was the appropriate network connectivity?

I've taken the liberty to modify the script - it's shown below, and also attached.  I'll confess I had to learn how to safely get the return code from the first command in a pipe (see **1).

Code Select Expand

#!/bin/sh

# Copyright (C) 2021 Franco Fichtner <franco@opnsense.org>
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
#    this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
# OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

LOCKFILE="/tmp/pkg_upgrade.progress"
TEE="/usr/bin/tee -a"

: > ${LOCKFILE}

URL=$(opnsense-update -M)
POPT="-c4 -s1500"

HOST=${URL#*://}
HOST=${HOST%%/*}
IPV4=$(host -t A ${HOST} | head -n 1 | cut -d\  -f4)
IPV6=$(host -t AAAA ${HOST} | head -n 1 | cut -d\  -f5)

echo "***GOT REQUEST TO AUDIT CONNECTIVITY***" >> ${LOCKFILE}
echo "Currently running $(opnsense-version) at $(date)" >> ${LOCKFILE}
if [ -n "${IPV4}" -a -z "${IPV4%%*.*}" ]; then
        echo "Checking connectivity for host: ${HOST} -> ${IPV4}" | ${TEE} ${LOCKFILE}
        exec 4>&1
        PRES=`{ { ping ${POPT} ${IPV4} 2>&1; printf $? 1>&3; } | ${TEE} ${LOCKFILE} 1>&4; } 3>&1`
        [ $PRES = 0 ] && echo "Checking connectivity for repository (IPv4): ${URL}" | ${TEE} ${LOCKFILE}
        [ $PRES = 0 ] && (pkg -4 update -f 2>&1) | ${TEE} ${LOCKFILE}
else
        echo "No IPv4 address could be found for host: ${HOST}" | ${TEE} ${LOCKFILE}
fi
if [ -n "${IPV6}" -a -z "${IPV6%%*:*}" ]; then
        echo "Checking connectivity for host: ${HOST} -> ${IPV6}" | ${TEE} ${LOCKFILE}
        exec 4>&1
        PRES=`{ { ping6 ${POPT} ${IPV6} 2>&1; printf $? 1>&3; } | ${TEE} ${LOCKFILE} 1>&4; } 3>&1`
        [ $PRES = 0 ] && echo "Checking connectivity for repository (IPv6): ${URL}" | ${TEE} ${LOCKFILE}
        [ $PRES = 0 ] && (pkg -6 update -f 2>&1) | ${TEE} ${LOCKFILE}
else
        echo "No IPv6 address could be found for host: ${HOST}" | ${TEE} ${LOCKFILE}
fi
echo '***DONE***' >> ${LOCKFILE}

I've also installed the new script on my firewall and it performs in my IPV4-only world just like the original, except that the database file doesn't get deleted.  I cannot check in an IPV6 environment, but visual inspection says there's a good chance it should work.  :D

**1
https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another

HTH

Peter

Hi Franco

Try to poke the update again. If it isn't a persistent problem it should work on a subsequent try.. make sure to account for the reboot that's going to happen once the base and kernel are being picked up.

I tried the update again and as you anticipated (I think) it said there were updates for the base and kernel packages.  These appeared to get installed ok but with no status ouput to the log window.  The system rebooted ok.

After it had rebooted, I did a set of audit tests:
CONNECTIVITY Audit was ok for IPV4 (IPV6 is not available from my ISP)
Then I checked for updates, and it reported all packages were up to date
However, the HEALTH Audit produced some very strange results (file attached) with the final package check reporting

Code Select Expand

.
.
Checking packages: .
zip-3.0_1 has no upstream equivalent
***DONE***for every single package (67).
I also noticed that the list of plugins was truncated to only those that I'd installed.
I checked for updates again, the plugin list was fully populated, and a subsequent HEALTH audit then looked normal.
I'm a bit puzzled by the "failed" HEALTH audit, but everything appears to be working fine.

Thanks for your help

PeterF

[both]

Hi

I thought I'd just tell you that I've been running a VPN between two instances of OPNsense successfully for some years.  Both sites use domestic-grade ADSL and cope with dynamic IP address allocation.  Moreover, I've used IPSEC to provide the VPN as a bridge between the two sites.

IPSEC is more challenging, because the configuration files that OPNsense generates ( /usr/local/etc/ipsec.conf and /usr/local/etc/ipsec.secrets ) both contain hard-coded ip addresses of both sites.  I've got round this by running a check script (every minute) to check VPN status, and the config files.  All discrepancies are fixed with sed and ipsec gets nudged to reload &/or reconnect.

I run the same script at both ends and it's been very successful.
If you want any more detail, then DM me.

PeterF

Hi

I upgraded my local FW to 22.1.5 a couple of days ago and that went fine.  Then I started the same upgrade on my remote FW, and that didn't complete.  It appears to have failed getting base-22.1.5-amd64.txz, and then didn't get the new kernel.  It also hasn't rebooted, yet.  It won't be long before there's a brownout which'll cause a reboot. The end of the update log (full copy attached) is here:

Code Select Expand

.
.
/var/cache/pkg/py38-charset-normalizer-2.0.12.txz
The cleanup will free 59 MiB
Deleting files: .......... done
All done
Nothing to do.
Starting web GUI...done.
Generating RRD graphs...done.
Fetching base-22.1.5-amd64.txz: ..............................pgrep:
Cannot open pidfile `/tmp/opnsense-fetch.pid.8WIINU': No such file or directory
[fetch: transfer timed out] failed, no signature found
***DONE***

How should I go about finishing off this update safely (& remotely) please?

TIA

PeterF

Hi

I experienced very similar problems to that of the OP.  I got my web-console text output back by restarting my browser (Firefox 89.0.1 on Win10).  I did this with the button on the about:profiles page (Firefox only).  I did nothing to clear the cache, or the cookies.

HTH

PeterF