update oddities

[skywalker007]

Hi,
for a while already I have the problem that one of my firewalls won't update via UI anymore.
It always resonds with

Code: [Select]

"Timeout while connecting to the selected mirror."Updating from shell works. Though it throws a warning:

Code: [Select]

Fetching change log information, please wait... fetch: transfer timed out
fetch: /tmp/changelog/changelog.txz.sig appears to be truncated: 0/1332 bytes
Checking that folder, it is indeed empty:

Code: [Select]

root@OPNsensemil:~ # ls -la /tmp/changelog/
total 8
drwxr-xr-x  2 root  wheel   512 Feb 10 13:01 .
drwxrwxrwt  6 root  wheel  1024 Feb 10 12:59 ..
root@OPNsensemil:~ #
Any advise how to fix this?
I am on 21.1, but this problem existed before.

Update:
this seems to be similar to this:
https://forum.opnsense.org/index.php?topic=21087.msg98506#msg98506

thanks, Till

[franco]

Proxy in your network? Changelogs are never empty...


Cheers,
Franco

[skywalker007]

No proxy

[franco]

# fetch -v https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/sets/changelog.txz.sig

Something is snatching binary data from your downloads...


Cheers,
Franco

[skywalker007]

This works:

Code: [Select]

root@OPNsensemil:~ # fetch -v https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/sets/changelog.txz.sig
resolving server address: pkg.opnsense.org:443
SSL options: 82004854
Peer verification enabled
Using CA cert file: /usr/local/etc/ssl/cert.pem
Verify hostname
TLSv1.2 connection established using ECDHE-RSA-CHACHA20-POLY1305
Certificate subject: /CN=pkg.opnsense.org
Certificate issuer: /C=US/O=Let's Encrypt/CN=R3
requesting https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/sets/changelog.txz.sig
remote size / mtime: 1332 / 1612887565
changelog.txz.sig                                     1332  B 3142 kBps    00sThough I just figured out that this box is very slow in resolving names. I don't know yet why. Question is what the UI updater does different from the cmdline updater. Different timeouts?
I'll try to fix DNS first.

[hruska]

This works:

Code: [Select]

root@OPNsensemil:~ # fetch -v https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/sets/changelog.txz.sig
resolving server address: pkg.opnsense.org:443
SSL options: 82004854
Peer verification enabled
Using CA cert file: /usr/local/etc/ssl/cert.pem
Verify hostname
TLSv1.2 connection established using ECDHE-RSA-CHACHA20-POLY1305
Certificate subject: /CN=pkg.opnsense.org
Certificate issuer: /C=US/O=Let's Encrypt/CN=R3
requesting https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/sets/changelog.txz.sig
remote size / mtime: 1332 / 1612887565
changelog.txz.sig                                     1332  B 3142 kBps    00sThough I just figured out that this box is very slow in resolving names. I don't know yet why. Question is what the UI updater does different from the cmdline updater. Different timeouts?
I'll try to fix DNS first.

Did you ever resolve this?  I am having the exact same issue.  Name resolution is also slow, but ends up working.  The UI updater still times out though.

[skywalker007]

Yes, solved. It is DNS configuration related. Though I am not sure what exactly made the difference. Have you checked "prefer IPv4 over IPv6" in DNS? If yes, can you disable it and test again?

[franco]

21.7 will gain a connectivity audit on the firmware page to make it easier to spot these kind of issues in the future.


Cheers,
Franco

[Cerberus]

If you have IPv6, check if connectivity is okay. Some of the mirros support IPv6 connectivity, if you have a broken IPv6 then you get timeouts because the updater is not falling back to IPv4.

[vlorentz]

Hello everyone,

I am facing a very similar issue in an HA environment.

The MASTER Firewall is always working well: NTP, DNS over TLS, Update, Plugin installation, etc...

However, the BACKUP firewall is extremely slow to resolve DNS and if I try to update the firmware, it took around 1h30 from 21.1.6 to 21.1.7!!! The MASTER took less than 10 minutes, including reboot time.

I am using 2 identical DEC690 appliances. I have only DNS over TLS configured (forcing all the traffic through the DNS over TLS, i.e., I am rerouting the TCP/UDP traffic to port 53 of the appliances to the firewall itself to answer these requests. I have no DNS entry under System --> Settings --> General --> DNS servers

I have found out that in some situations (not related to HA and CARP), the firewall can be brought in situations in which I cannot log in again, since the time is wrong (reset to April 2017, for example when the appliance was turned off). It seems to be that NTP is trying to use the DNS servers in these general settings, and not the ones configured in Unbound DNS (i.e., DoT).

Does anybody have an idea, wehre I could search to solve the issue of the extremely slow BACKUP appliance?

Nota: if I disconnect the MASTER and the BACKUP becomes MASTER itself, the problem is transferred. The problem is always on the BACKUP, never on the MASTER (no physical relation).

One more thing: I am on a single WAN (but using CARP VIP on WAN and on LAN side).
And the Network Time always shows "No active peers available " on the BACKUP. Also the proofpoint Telemetry status always shows a cross (i.e., not connected) on the BACKUP.

And if I do a DNS Lookup on the BACKUP (after having disabled the blocking DNS firewall rules and inserted 1.1.1.1 in the general DNS servers (i.e., not Unbound DNS)), I get the following:
Server    Query time
127.0.0.1    475 msec
1.1.1.1    No response

Same on the MASTER:
Server    Query time
127.0.0.1    181 msec
1.1.1.1    51 msec

[vlorentz]

I found the origin of my problem, why the BACKUP was so slow to check for firmware updates in the GUI (and it also solved all the DNS stuff I explained in my previous post).

The issue is related to the manual Outbound NAT rules I have added to support the VIP on the WAN interface.

The configuration that makes problems is (same on MASTER and BACKUP):
Interface: ETH0_WAN
TCP/IP Version: IPv4
Source Address: any
Translation / target: 192.168.0.250 (Shared WAN CARP Virtual IP)

When I change the configuration to the following:
Interface: ETH0_WAN
TCP/IP Version: IPv4
Source Address: ETH1_LAN net
Translation / target: 192.168.0.250 (Shared WAN CARP Virtual IP)

then I am getting DNS answers on both MASTER and BACKUP.

This means, that I have to generate 1 Outbound NAT rule per LAN or VLAN, and that having only one single rule to manage all the LAN and VLANs does not work properly. Most probably I am missing something, but I am pretty sure that the original automatic rule was a single rule regrouping all the LAN and VLAN interfaces.

I would be curious to hear the explanation.
Thank you.