Happy Monday! When I check for new updates, it hands for a very long time showing:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7.7_3 at Mon Oct 30 10:25:03 CET 2023
Fetching changelog information, please wait... fetch: transfer timed out
fetch: /usr/local/opnsense/changelog/changelog.txz appears to be truncated: 0/117388 bytes
Updating OPNsense repository catalogue...
I'm using the (default) mirror
Anyone else??
IPv6 being defunct biting you? Try the connectivity audit and if you don't need IPv6 make sure it's globally disabled.
Cheers,
Franco
Quote from: franco on October 30, 2023, 10:48:52 AM
IPv6 being defunct biting you? Try the connectivity audit and if you don't need IPv6 make sure it's globally disabled.
Cheers,
Franco
Thanks Franco - but up until now, updates always worked - and I haven't introduced anything new (never configured IPv6) just installed with defaults way back when I installed 21.1 initially - Not sure what you mean with "connectivity audit" ?!?
Same problem for me I think.
Mine could be different:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7.1_3 at Mon Oct 30 07:10:38 CDT 2023
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Child process pid=55281 terminated abnormally: Bus error
Child process pid=55452 terminated abnormally: Bus error
Child process pid=56205 terminated abnormally: Bus error
self: No packages available to install matching 'opnsense'
***DONE***
Something is really broken - after performing a reboot and everything starting just fine - when I simply check "System-> Firmware-> Status" - it's blank and there is a busy rotating circle on the tab Status showing that something is happening - but nothing shows up. When I click on "plugins" - they are all blank ?!? Then I click on Packages tab or Change Log tab and it shows the contents . . . The Status tab stops the spinning wheel and shows the content as expected . . . very odd behavior!! I could roll back to 23.7.6 from a previous backup - but I'm pretty sure up until the 23.7.7_3 update. everything was fine!
Browser cache?
Cheers,
Franco
Quote from: rhubarb on October 30, 2023, 01:19:19 PM
Mine could be different:
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7.1_3 at Mon Oct 30 07:10:38 CDT 2023
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Child process pid=55281 terminated abnormally: Bus error
Child process pid=55452 terminated abnormally: Bus error
Child process pid=56205 terminated abnormally: Bus error
self: No packages available to install matching 'opnsense'
***DONE***
Disk or RAM dying.
Cheers,
Franco
Quote from: franco on October 30, 2023, 04:39:10 PM
Browser cache?
Cheers,
Franco
Nice guess - but no dice! ::) Cleared the cache and didn't help - also tried different browsers to make sure. now all tabs are blank and the status just spins :-\
And a different browser will not help either?
It could still be trying to process "something"... for the hammer approach:
# killall pkg
# killall pkg-static
... and reassess
Cheers,
Franco
Quote from: franco on October 30, 2023, 05:19:09 PM
And a different browser will not help either?
It could still be trying to process "something"... for the hammer approach:
# killall pkg
# killall pkg-static
... and reassess
Cheers,
Franco
Sorry, tried what you suggested - still no improvement. I've been running OPNSense now since 21.1 (over a year) and many updates later - The Update process has been stable as a rock until now . . . Why is the changelog.txz getting truncated?!?
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7.7_3 at Mon Oct 30 17:39:52 CET 2023
Fetching changelog information, please wait... fetch: transfer timed out
fetch: /usr/local/opnsense/changelog/changelog.txz appears to be truncated: 0/117388 bytes
Updating OPNsense repository catalogue...
Checking integrity... done (0 conflicting)
Your packages are up to date.
Checking for upgrades (1 candidates): . done
Processing candidates (1 candidates): . done
Checking integrity... done (0 conflicting)
Your packages are up to date.
Because there is an issue with the download?
You can try the clean fetch method:
# fetch https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/sets/changelog.txz
As said before on such reports the download even sees the length of the file (117388) reported by the server but goes on to download 0 bytes instead...
Cheers,
Franco
Tried the fetch command from the shell - it appears to hang or REALLY slow - if I wait long enough - I get:
root@FoxOPN:/home/rfox # fetch https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/sets/changelog.txz
changelog.txz 114 kB 3101 kBps 00s
So something with connectivity to your mirror?!?
Well how long does this run? The fetch is normally on a 30 seconds timeout I think... plenty of time to download the file.
I don't think it's the mirror directly, but something from your OPNsense to the mirror.
Cheers,
Franco
Quote from: rfox on October 30, 2023, 05:57:40 PM
114 kB 3101 kBps 00s
That's not slow. Seems more like the download just takes a long time to start. Maybe a DNS issue. Or, as franco suggested, broken IPv6. Did you run the connectivity audit? (System: Firmware: Status: Run an audit: Connectivity)
Cheers
Maurice
Here's the results . . .
***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 23.7.7_3 at Tue Oct 31 10:22:35 CET 2023
Checking connectivity for host: pkg.opnsense.org -> 89.149.222.99
PING 89.149.222.99 (89.149.222.99): 1500 data bytes
1508 bytes from 89.149.222.99: icmp_seq=0 ttl=55 time=12.137 ms
1508 bytes from 89.149.222.99: icmp_seq=1 ttl=55 time=12.140 ms
1508 bytes from 89.149.222.99: icmp_seq=2 ttl=55 time=12.139 ms
1508 bytes from 89.149.222.99: icmp_seq=3 ttl=55 time=12.227 ms
--- 89.149.222.99 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.137/12.161/12.227/0.038 ms
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 851 packages processed.
Updating SunnyValley repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ... done
Processing entries: ....... done
SunnyValley repository update completed. 64 packages processed.
All repositories are up to date.
Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:5300:a010:1::1
PING6(1548=40+8+1500 bytes) 2a01:598:8982:a072:a043:bff:fed0:5d49 --> 2001:1af8:5300:a010:1::1
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=0 hlim=49 time=149.033 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=1 hlim=49 time=55.866 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=2 hlim=49 time=55.509 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=3 hlim=49 time=51.742 ms
--- 2001:1af8:5300:a010:1::1 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 51.742/78.038/149.033/41.021 ms
Checking connectivity for repository (IPv6): https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
Updating OPNsense repository catalogue...
Sits here for a long time . . . seems hung ?!?
Checked the logs:
2023-10-31T10:20:24 Error configd.py Timeout (120) executing : firmware remote
2023-10-31T10:18:22 Error configd.py Timeout (120) executing : firmware tiers
2023-10-31T07:09:24 Error configd.py [33319d67-e555-49cf-9475-ca30ebe54b50] returned exit status 1
2023-10-30T17:48:58 Error configd.py [4881b3d8-f351-4c31-86c2-78729c65a82b] Script action failed with Command '/usr/local/opnsense/scripts/firmware/query.sh remote ' returned non-zero exit status 143. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/actions/script_output.py", line 44, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/firmware/query.sh remote ' returned non-zero exit status 143.
2023-10-30T17:48:58 Error configd.py [113edd75-e752-4bfe-b3f0-936e70a37559] Script action failed with Command '/usr/local/opnsense/scripts/firmware/query.sh remote ' returned non-zero exit status 143. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/actions/script_output.py", line 44, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/firmware/query.sh remote ' returned non-zero exit status 143.
2023-10-30T17:48:53 Error configd.py [80265804-0d0e-430c-8134-0da4bf5e010d] Script action failed with Command '/usr/local/opnsense/scripts/firmware/query.sh tiers ' returned non-zero exit status 143. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/actions/script_output.py", line 44, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/firmware/query.sh tiers ' returned non-zero exit status 143.
2023-10-30T17:48:53 Error configd.py [3663a634-0cc7-45b9-a3c4-164e9dfa625a] Script action failed with Command '/usr/local/opnsense/scripts/firmware/query.sh tiers ' returned non-zero exit status 143. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/actions/script_output.py", line 44, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/firmware/query.sh tiers ' returned non-zero exit status 143.
root@FoxOPN:/home/rfox # ps aux | grep pkg
root 8151 0.0 0.1 22072 7144 - I 11:54 0:00.01 pkg -6 update -f
root 8600 0.0 0.0 12624 1656 - IC 11:54 0:00.00 /usr/bin/tee -a /tmp/pkg_upgrade.progress
root 8808 0.0 0.1 22076 7476 - I 11:54 0:00.01 pkg -6 update -f
root 60335 0.0 0.1 22072 7148 - I 12:00 0:00.01 pkg update -f
root 60774 0.0 0.1 24772 9392 - I 12:00 0:00.03 pkg update -f
root 78912 0.0 0.0 12872 1944 - I 11:54 0:00.00 /usr/local/bin/flock -n -o /tmp/pkg_upgrade.progress /usr/local/opnsense/scripts/firmware/connection.sh
root 99288 0.0 0.0 12724 1936 1 S+ 12:02 0:00.00 grep pkg
10 Minutes later - and still hanging - I might try to revert back to 27.7.6 and see if that fixes it . .
***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 23.7.7_3 at Tue Oct 31 10:22:35 CET 2023
Checking connectivity for host: pkg.opnsense.org -> 89.149.222.99
PING 89.149.222.99 (89.149.222.99): 1500 data bytes
1508 bytes from 89.149.222.99: icmp_seq=0 ttl=55 time=12.137 ms
1508 bytes from 89.149.222.99: icmp_seq=1 ttl=55 time=12.140 ms
1508 bytes from 89.149.222.99: icmp_seq=2 ttl=55 time=12.139 ms
1508 bytes from 89.149.222.99: icmp_seq=3 ttl=55 time=12.227 ms
--- 89.149.222.99 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.137/12.161/12.227/0.038 ms
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 851 packages processed.
Updating SunnyValley repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ... done
Processing entries: ....... done
SunnyValley repository update completed. 64 packages processed.
All repositories are up to date.
Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:5300:a010:1::1
PING6(1548=40+8+1500 bytes) 2a01:598:8982:a072:a043:bff:fed0:5d49 --> 2001:1af8:5300:a010:1::1
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=0 hlim=49 time=149.033 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=1 hlim=49 time=55.866 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=2 hlim=49 time=55.509 ms
1508 bytes from 2001:1af8:5300:a010:1::1, icmp_seq=3 hlim=49 time=51.742 ms
--- 2001:1af8:5300:a010:1::1 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 51.742/78.038/149.033/41.021 ms
Checking connectivity for repository (IPv6): https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
Updating OPNsense repository catalogue...
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/latest/meta.txz: Operation timed out
repository OPNsense has no meta file, using default settings
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/latest/packagesite.pkg: Operation timed out
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/latest/packagesite.txz: Operation timed out
Unable to update repository OPNsense
Updating SunnyValley repository catalogue...
pkg: https://updates.zenarmor.com/opnsense/FreeBSD:13:amd64/23.7/latest/meta.txz: Operation timed out
repository SunnyValley has no meta file, using default settings
pkg: https://updates.zenarmor.com/opnsense/FreeBSD:13:amd64/23.7/latest/packagesite.pkg: Operation timed out
penguin@mypc:/tmp$ curl https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/">here</a>.</p>
</body></html>
penguin@mypc:/tmp$ wget https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
--2023-10-31 10:55:03-- https://pkg.opnsense.org/FreeBSD:13:amd64/23.7
Resolving pkg.opnsense.org (pkg.opnsense.org)... 89.149.222.99, 2001:1af8:5300:a010:1::1
Connecting to pkg.opnsense.org (pkg.opnsense.org)|89.149.222.99|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/ [following]
--2023-10-31 10:55:04-- https://pkg.opnsense.org/FreeBSD:13:amd64/23.7/
Reusing existing connection to pkg.opnsense.org:443.
HTTP request sent, awaiting response... 200 OK
Length: 1289 (1.3K) [text/html]
Saving to: '23.7'
23.7 100%[=================================================>] 1.26K --.-KB/s in 0s
2023-10-31 10:55:04 (195 MB/s) - '23.7' saved [1289/1289]
All good here and using a work vpn that puts me in a different geographic location.
Maybe some geo thing if the mirror is behind a cdn ?
It still looks like flaky IPv6. Maybe better turn it off.
Cheers,
Franco
Quote from: franco on October 31, 2023, 12:00:12 PM
It still looks like flaky IPv6. Maybe better turn it off.
Cheers,
Franco
Thx Franco - will try. Strange that it always worked before 27.7.7_3 ?!? So, how do I turn off IPv6 completely?
Start with your WAN IPv6 mode... it probably still is DHCPv6 from the default setting?
Cheers,
Franco
Turned off DHCPv6 on WAN side (was default) and noticed on the LAN was "Track Interface" turned on ?!? Turned it off. Using default mirror still failed (couldn't figure out how to release the IPv6 address from console - but oddly enough, I switched to a different mirror "C0urier.net" in Sweden - and because it doesn't have IPv6 everything works as expected.
My question is, when using the Default mirror - why can't it fall back to IPv4 when IPv6 times out? Something has changed since last update to cause this to fail - was there a script change maybe?
Anyone else affected? Turning off IPv6 is not really a good solution . . .
Thanks for your help!
If the connection appears to contact the server successfully but stalls on the actual download, how can anyone blame IPv6 functionality itself at that point? IPv4 does exactly the same in these situations but it's generally less prone to such intermittent connectivity issues :)
And I'm not sure why you want IPv6 enabled when your connection isn't working as it should as proven here. Ask your provider...
Cheers,
Franco
Quote from: franco on October 31, 2023, 01:49:36 PM
If the connection appears to contact the server successfully but stalls on the actual download, how can anyone blame IPv6 functionality itself at that point? IPv4 does exactly the same in these situations but it's generally less prone to such intermittent connectivity issues :)
And I'm not sure why you want IPv6 enabled when your connection isn't working as it should as proven here. Ask your provider...
Cheers,
Franco
Understood. Just for clarity, I did not enable IPv6 on the WAN - it was default set that way. All I'm saying is - for almost a year of using OPNSense - I have had NO issue with this - then, all of a sudden - after updating to 27.7._3 it started to act real weird. So, something obviously changed in the latest update to cause this issue.
Thx again!
It could be coincidence. Your ISP may have changed something, started rolling out partial IPv6. It has happened a number of times over the years. Usually users see the system mostly on updates and this is when they notice things may not be working as before. To put it in a funny way: "blame update first, reassess situation later" :)
The choice here is to either support IPv6 out of the box for OPNsense or not. For most this will be better than having to enable it manually (more are less likely to even attempt it). It also certainly helps with testing on our end to have this as a standard.
Simply trying to give more context here. I know how difficult the situation is.
Cheers,
Franco
Excellent reply! ;D I will check with my provider to see if they changed something recently.
Yet, if I turn off IPv6 and it works fine with a different mirror than the "Default" mirror - that says a lot as well. I plan to revert to a previous backup 27.7.6 just to see if the issue goes away - that would be more definitive for us!
Thx again and Happy Halloween on top!!