pkg, fetch & update check issues after updating to 22.1

[survive]

Hi there,

I have a similar thread going here: https://forum.opnsense.org/index.php?topic=25725.0

but figured I would post here in the hopes of figuring out the problem before 22.1 ships.

Long-time OPNsense user. I started having problems updating to 21.7.6. Posted but didn't get traction.

I just did my revert-the-vm process back to 21.7.1 and updated to OPNsense 22.1.b_146.

Same issues:

Code: [Select]

***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 22.1.b_146 (amd64/OpenSSL) at Wed Jan 12 23:31:50 CST 2022
Checking connectivity for host: pkg.opnsense.org
PING 89.149.211.205 (89.149.211.205): 56 data bytes
64 bytes from 89.149.211.205: icmp_seq=0 ttl=49 time=105.870 ms
64 bytes from 89.149.211.205: icmp_seq=1 ttl=49 time=106.065 ms
64 bytes from 89.149.211.205: icmp_seq=2 ttl=49 time=105.280 ms
64 bytes from 89.149.211.205: icmp_seq=3 ttl=49 time=106.036 ms

--- 89.149.211.205 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 105.280/105.813/106.065/0.317 ms
PING6(56=40+8+8 bytes) 2600:1700:5db0:6050:207:e9ff:fe18:beef --> 2001:1af8:4f00:a005:5::
16 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=0 hlim=45 time=107.744 ms
16 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=1 hlim=45 time=107.511 ms
16 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=2 hlim=45 time=107.520 ms
16 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=3 hlim=45 time=107.336 ms

--- 2001:1af8:4f00:a005:5:: ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 107.336/107.527/107.744/0.145 ms
Checking connectivity for URL: https://pkg.opnsense.org/FreeBSD:12:amd64/21.7
Updating OPNsense repository catalogue...
Fetching meta.txz: . done
pkg: https://pkg.opnsense.org/FreeBSD:12:amd64/21.7/latest/packagesite.txz: Operation timed out
Unable to update repository OPNsense
Updating SunnyValley repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.txz: .. done
Processing entries: .... done
SunnyValley repository update completed. 31 packages processed.
Error updating repositories!
***DONE***

I don't get it. I can reach the hosts, but I can't get anything from them!

-Will

[franco]

Hi Will,

As stated in the other forum post it might be a fragmentation issue. Since I can see you use a VM some hosts have had issues with full packet size. Try lowering your WAN interface MTU to e.g. 1400 to test that theory and if it works work your way up to the maximum value that works.


Cheers,
Franco

[iamperson347]

I experienced the same issue with updates in a VM but with physical nics passed through (no virtual nic). Does this rule out fragmenation due to running in a VM?

[franco]

Not entirely. From the command line I would try to see how the oversize ping goes:

# ping -c4 -s1500 pkg.opnsense.org
# ping6 -c4 -s1500 pkg.opnsense.org

Of course it's difficult to tell if pkg utility will use IPv4 or IPv6. As a rule of thumb it should prefer IPv6 unless that has been turned off or IPv4 is preferred (system: settings: general). And you need to use the mirror you set up if it's not the default.


Cheers,
Franco

[iamperson347]

Code: [Select]

root@OPNsense:~ #  ping -c4 -s1500 pkg.opnsense.org
PING6(1548=40+8+1500 bytes) 2600:8805:7f20:200:f0c0:8e63:4c48:70d3 --> 2001:1af8:4f00:a005:5::
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=0 hlim=51 time=100.618 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=1 hlim=51 time=98.166 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=2 hlim=51 time=98.463 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=3 hlim=51 time=98.095 ms

--- pkg.opnsense.org ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 98.095/98.835/100.618/1.038 ms
root@OPNsense:~ #
root@OPNsense:~ #
root@OPNsense:~ # ping6 -c4 -s1500 pkg.opnsense.org
PING6(1548=40+8+1500 bytes) 2600:8805:7f20:200:f0c0:8e63:4c48:70d3 --> 2001:1af8:4f00:a005:5::
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=0 hlim=51 time=98.541 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=1 hlim=51 time=98.151 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=2 hlim=51 time=99.409 ms
1508 bytes from 2001:1af8:4f00:a005:5::, icmp_seq=3 hlim=51 time=99.212 ms

--- pkg.opnsense.org ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 98.151/98.828/99.409/0.506 ms
root@OPNsense:~ #
root@OPNsense:~ #
root@OPNsense:~ # ping -c4 -s1500 mirror.dns-root.de
PING6(1548=40+8+1500 bytes) 2600:8805:7f20:200:f0c0:8e63:4c48:70d3 --> 2606:4700:3036::ac43:ce5d
1508 bytes from 2606:4700:3036::ac43:ce5d, icmp_seq=0 hlim=58 time=13.659 ms
1508 bytes from 2606:4700:3036::ac43:ce5d, icmp_seq=1 hlim=58 time=13.070 ms
1508 bytes from 2606:4700:3036::ac43:ce5d, icmp_seq=2 hlim=58 time=11.723 ms
1508 bytes from 2606:4700:3036::ac43:ce5d, icmp_seq=3 hlim=58 time=12.884 ms

--- mirror.dns-root.de ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 11.723/12.834/13.659/0.702 ms
root@OPNsense:~ #
root@OPNsense:~ # ping6 -c4 -s1500 mirror.dns-root.de
PING6(1548=40+8+1500 bytes) 2600:8805:7f20:200:f0c0:8e63:4c48:70d3 --> 2606:4700:3034::6815:16b3
1508 bytes from 2606:4700:3034::6815:16b3, icmp_seq=0 hlim=58 time=12.176 ms
1508 bytes from 2606:4700:3034::6815:16b3, icmp_seq=1 hlim=58 time=12.748 ms
1508 bytes from 2606:4700:3034::6815:16b3, icmp_seq=2 hlim=58 time=13.794 ms
1508 bytes from 2606:4700:3034::6815:16b3, icmp_seq=3 hlim=58 time=12.641 ms

--- mirror.dns-root.de ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 12.176/12.840/13.794/0.591 ms
root@OPNsense:~ #
root@OPNsense:~ #
root@OPNsense:~ # ping -4 -c4 -s1500 mirror.dns-root.de
PING mirror.dns-root.de (172.67.206.93): 1500 data bytes
1508 bytes from 172.67.206.93: icmp_seq=0 ttl=59 time=13.823 ms
1508 bytes from 172.67.206.93: icmp_seq=1 ttl=59 time=14.634 ms
1508 bytes from 172.67.206.93: icmp_seq=2 ttl=59 time=13.123 ms
1508 bytes from 172.67.206.93: icmp_seq=3 ttl=59 time=12.400 ms

--- mirror.dns-root.de ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.400/13.495/14.634/0.828 ms
root@OPNsense:~ #

----

My update issue occurs right after upgrading from latest community to dev build. It seems I can't even try to get on 22.1rc1 due to the issues with fetch and pkg after switching to dev.

On community, no issues with updates. If I roll back my VM, no issues.

It doesn't seem to matter which mirror I select. In fact, fetch seems to act up with any url I throw it, and it is not a DNS issue from what I can see.

[lrosenman]

I was able to get around this problem by doing:

pkg -4 update
pkg -4 upgrade

and rebooting.

I wonder if there is a way to force pkg -4 ?

[iamperson347]

I think this only partially upgrades the firewall.

With that being said, you can actually set the firewall to prefer ipv4. I did this and the update process worked again. Now that I'm on 22.1rc1, I unset the prefer ipv4 option, rebooted to be safe, and update checks are still working fine.

I'm wondering if there was just some issue with pkg/fetch and ipv6 connectivity on some of the dev builds? I know ipv6 works fine for me on the latest community build and it now seems fine again on 22.1rc1.

[lrosenman]

Thanks for that hint, I did the prefer 4 over 6 and it updated to:
OPNsense 22.1.r1-amd64


from r_3.  So, yes, there were changes.

Now I've turned off the preference, and update checks, etc all work.

Interesting.

Thanks for the hint.

[iamperson347]

NP - it was your other post that prompted me to test it.

I wonder if franco has any thoughts on why this happened to some of us. Either way, I'm glad it's fixed.

[franco]

No idea, but it probably revolves around IPv6 use of some sort. Also, sometimes, local proxies or security features of ISPs can interfere with signature fetching (false positives in base64 encoded binary data).


Cheers,
Franco