[SOLVED] Attempts to update time out with RSS enabled

[clarknova]

OPNsense 23.1.6-amd64

I have not been able to get the latest update. I've tried every method I know of, but they all time out.

Web UI:

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.1.6 at Thu May 18 17:25:10 MDT 2023
Fetching changelog information, please wait... Missing /usr/local/etc/pkg/repos/OPNsense.conf
fetch: transfer timed out
Updating FreeBSD repository catalogue...
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:13:amd64/quarterly/meta.txz: Operation timed out
repository FreeBSD has no meta file, using default settings
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:13:amd64/quarterly/packagesite.pkg: Operation timed out
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:13:amd64/quarterly/packagesite.txz: Operation timed out
Unable to update repository FreeBSD
Error updating repositories!
pkg: Unknown repository: OPNsense
***DONE***

Code Select Expand

# /usr/local/opnsense/scripts/firmware/connection.sh
Checking connectivity for host: mirror.sfo12.us.leaseweb.net -> 209.58.135.187
PING 209.58.135.187 (209.58.135.187): 1500 data bytes
1508 bytes from 209.58.135.187: icmp_seq=0 ttl=58 time=45.525 ms
1508 bytes from 209.58.135.187: icmp_seq=1 ttl=58 time=45.178 ms
1508 bytes from 209.58.135.187: icmp_seq=2 ttl=58 time=45.760 ms
1508 bytes from 209.58.135.187: icmp_seq=3 ttl=58 time=45.671 ms

--- 209.58.135.187 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 45.178/45.534/45.760/0.222 ms
Checking connectivity for repository (IPv4): http://mirror.sfo12.us.leaseweb.net/opnsense/FreeBSD:13:amd64/23.1
Updating OPNsense repository catalogue...

Code Select Expand

# opnsense-bootstrap
This utility will attempt to turn this installation into the latest
OPNsense 23.1 release.  All packages will be deleted, the base
system and kernel will be replaced, and if all went well the system
will automatically reboot.

Proceed with this action? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:13:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
Installing pkg-1.19.1_1...
package pkg is already installed, forced install
Extracting pkg-1.19.1_1: 100%
Updating FreeBSD repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01   
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:13:amd64/quarterly/packagesite.pkg: Operation timed out
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:13:amd64/quarterly/packagesite.txz: Operation timed out
Unable to update repository FreeBSD
Error updating repositories!

Code Select Expand

# opnsense-update -bkp
Missing /usr/local/etc/pkg/repos/OPNsense.conf

I can ping the mirror, pkg.freebsd.org and any other internet host from the OPNsense shell no problem. LAN hosts have functioning internet. I'm not sure how I got here, but I suppose the next step is to install from USB with config recovery unless somebody has a better suggestion.

[clarknova]

Something has changed, but not for the better:

Code Select Expand

# /usr/local/opnsense/scripts/firmware/connection.sh
Missing /usr/local/etc/pkg/repos/OPNsense.conf
Usage: host [-aCdilrsTvw46] [-c class] [-N ndots] [-R number]
            [-t type] [-W wait] name [server]
-a same as -v -t ANY
-C query SOA records from all authoritative name servers
-c use this query class (IN, CH, HS, etc)
-d produce verbose output, same as -v
-i use IP6.INT for IPv6 reverse lookups
-l list records in a zone via AXFR
-N consider names with at least this many dots as absolute
-R retry UDP queries this many times
-r disable recursive query
-s do not ignore SERVFAIL responses
-T send query via TCP
-t use this query type (A, AAAA, MX, etc)
-v produce verbose output
-w wait forever for a server reply
-W wait this many seconds for a reply
-4 use IPv4 only
-6 use IPv6 only
Usage: host [-aCdilrsTvw46] [-c class] [-N ndots] [-R number]
            [-t type] [-W wait] name [server]
-a same as -v -t ANY
-C query SOA records from all authoritative name servers
-c use this query class (IN, CH, HS, etc)
-d produce verbose output, same as -v
-i use IP6.INT for IPv6 reverse lookups
-l list records in a zone via AXFR
-N consider names with at least this many dots as absolute
-R retry UDP queries this many times
-r disable recursive query
-s do not ignore SERVFAIL responses
-T send query via TCP
-t use this query type (A, AAAA, MX, etc)
-v produce verbose output
-w wait forever for a server reply
-W wait this many seconds for a reply
-4 use IPv4 only
-6 use IPv6 only
No IPv4 address could be found for host:
No IPv6 address could be found for host:

I didn't manually delete /usr/local/etc/pkg/repos/OPNsense.conf or any other file. Is my mSATA dying?

[clarknova]

I ran a health check:

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.1.6 at Thu May 18 17:43:45 MDT 2023
Fetching changelog information, please wait... Missing /usr/local/etc/pkg/repos/OPNsense.conf
fetch: transfer timed out
Updating FreeBSD repository catalogue...
Fetching meta.txz: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
FreeBSD repository update completed. 32881 packages processed.
All repositories are up to date.
pkg: Unknown repository: OPNsense
Updating database digests format: . done
Checking for upgrades (101 candidates): .......... done
Processing candidates (101 candidates): ...... done
The following 105 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
binutils: 2.40_4,1
brotli: 1.0.9,1
cairo: 1.17.4_2,3
dbus: 1.14.6,1
dejavu: 2.37_1
easy-rsa: 3.1.2
encodings: 1.0.5,1
font-bh-ttf: 1.0.3_4
font-misc-ethiopic: 1.0.4
font-misc-meltho: 1.0.3_4
fontconfig: 2.14.2,1
freetype2: 2.12.1_2
fribidi: 1.0.12
gcc12: 12.2.0_5
graphite2: 1.3.14
harfbuzz: 7.1.0
libICE: 1.1.0_1,1
libSM: 1.2.3,1
libX11: 1.7.2,1
libXau: 1.0.9
libXdmcp: 1.1.3
libXext: 1.3.4,1
libXft: 2.3.7
libXrender: 0.9.10_2
libdatrie: 0.2.13_1
libfontenc: 1.1.4
libglvnd: 1.6.0
libssh2: 1.10.0_1,3
libthai: 0.1.29
libxcb: 1.15_1
lua54: 5.4.4
metis: 5.1.0_9
mkfontscale: 1.2.1
mpc: 1.2.1
mpfr: 4.2.0,1
openblas: 0.3.20,1
pango: 1.50.9_1
pixman: 0.42.2
pkcs11-helper: 1.29.0
png: 1.6.39
suitesparse-amd: 3.0.3
suitesparse-camd: 3.0.3
suitesparse-ccolamd: 3.0.3
suitesparse-cholmod: 4.0.3
suitesparse-colamd: 3.0.3
suitesparse-config: 7.0.1
suitesparse-umfpack: 6.1.0
xorg-fonts-truetype: 7.7_1
xorgproto: 2022.1
zstd: 1.5.4

Installed packages to be UPGRADED:
flowd: 0.9.1_3 -> 0.9.1_4
glib: 2.76.1,2 -> 2.76.2,2
libxml2: 2.10.3_2 -> 2.10.4
openvpn: 2.5.8 -> 2.6.4
py39-duckdb: 0.6.1 -> 0.7.1
sqlite3: 3.41.0_2,1 -> 3.41.2,1
suricata: 6.0.9_1 -> 6.0.9_4
syslog-ng: 3.38.1 -> 4.1.1

Installed packages to be REINSTALLED:
ca_root_nss-3.89 (options changed)
curl-8.0.1 (options changed)
cyrus-sasl-2.1.28 (options changed)
cyrus-sasl-gssapi-2.1.28 (options changed)
dnsmasq-2.89_1,1 (options changed)
expat-2.5.0 (options changed)
gettext-runtime-0.21.1 (options changed)
iftop-1.0.p4 (options changed)
isc-dhcp44-relay-4.4.3P1 (options changed)
isc-dhcp44-server-4.4.3P1 (options changed)
krb5-1.20.1 (options changed)
ldns-1.8.3 (options changed)
libevent-2.1.12 (direct dependency removed: openssl)
libfido2-1.13.0 (options changed)
libiconv-1.17 (options changed)
libnet-1.2,1 (options changed)
libsodium-1.0.18 (options changed)
libunistring-1.1 (options changed)
lighttpd-1.4.69 (options changed)
lzo2-2.10_1 (options changed)
monit-5.33.0 (options changed)
mpd5-5.9_14 (options changed)
mpdecimal-2.5.1 (options changed)
nettle-3.8.1 (options changed)
ntp-4.2.8p15_5 (options changed)
oniguruma-6.9.8_1 (options changed)
openldap26-client-2.6.4 (options changed)
openssh-portable-9.2.p1,1 (options changed)
pcre-8.45_3 (options changed)
pcre2-10.42 (options changed)
py39-Babel-2.12.1 (options changed)
py39-Jinja2-3.1.2 (options changed)
py39-cryptography-3.4.8_1,1 (direct dependency changed: python39)
py39-dnspython-2.3.0,1 (options changed)
py39-netaddr-0.8.0 (options changed)
py39-numpy-1.24.1,1 (options changed)
py39-yaml-6.0 (options changed)
python39-3.9.16_2 (options changed)
readline-8.2.1 (options changed)
rrdtool-1.8.0_2 (options changed)
samplicator-1.3.8.r1_1 (options changed)
squid-5.8 (options changed)
strongswan-5.9.10_1 (options changed)
sudo-1.9.13p3 (options changed)
unbound-1.17.1_2 (options changed)
wpa_supplicant-2.10_6 (options changed)
zip-3.0_1 (options changed)

Number of packages to be installed: 50
Number of packages to be upgraded: 8
Number of packages to be reinstalled: 47

The process will require 687 MiB more space.
207 MiB to be downloaded.
self: No packages available to install matching 'opnsense'
***DONE***

[clarknova]

I just did a fresh install with a config import and it still won't update. Apparently there's something wrong with my config, but I'm scratching my head. I've managed dozens of OPNsense installs and never seen this problem. The only thing I can think that I've done unique on this system is to enable RSS, so maybe that's a problem. I'll have to try disabling it.

[clarknova]

Bingo. I changed `net.inet.rss.enabled` from `1` to `0` and rebooted. OPNsense updates fine now.

[franco]

> Missing /usr/local/etc/pkg/repos/OPNsense.conf

This is strange as the file is created during boot and other situations... manually invoked via:

# configctl firmware configure

or

# /usr/local/etc/rc.configure_firmware

if you fancy a bit of output.


Cheers,
Franco

[clarknova]

I was seeing many strange symptoms, like sometimes I got that error and sometimes not. I tried using 'cp', 'cat', 'dd' and 'echo' to populate that file from another system but got an error every time. Something like "file does not exist" or similar.

It also required a lot of trial and error just to get OPNsense to resolve names properly. If I tried to ping an internet host by name it would just time out with a DNS error, even though on a packet dump I could see DNS requests going out and responses coming back. I eventually got it to work by disabling Unbound and telling OPNsense to not use the internal resolver. It's strange that just changing that one tunable fixed these problems.