OPNSense Update - No route to host

[jimbeam128]

Hi,

I´ve got a weird Issue updating my OPNSense Backup Appliance. I Use Version 20.1.7 and would like to upgrade.

When I Hit "Click to check for updates" I get the response  "Timeout while connecting to the selected mirror".

What really strange is, the Master-Appliance is able to fetch updates.

The configuration of the two appliances is identical. I even cloned the Master-VM and reconfigured the clone to be the backup-VM, to be sure.

Same Networks, same Gateways (different IP´s ;-) ) and so on.

A ping to external (for example google.com) is possible.

When I try to connect to external (from console) with a tcp-connection (for example ftp / http) I get the response "unable to connect to remote host".

Both systems are behind a sophos UTM firewall where the same rules apply to both systems.

Packets from the master-vm arrive at the UTM. Packets from the backup-appliance do not arrive at the UTM.

This is the fw-log output from the backup-appliance.

LANInternal <- [timestamp] source is LAN-Address destination is target-address prot is tcp label is "let out anything from firewall host itsel (force gw)


I read in different forum-topics that several users seem to have the same problem... - but I found no solution...

[franco]

Can you try this on the broken machine and paste the output here?

# pkg -d update -f

Beware of subtleties with defunct IPv6 which is probably hard to spot when DNS names are printed for connection.


Cheers,
Franco

[jimbeam128]

Hi Franco, thanks for your reply!

here´s the output:

root@OPNsense02:~ # pkg -d update -f
DBG(1)[65636]> pkg initialized
Updating OPNsense repository catalogue...
DBG(1)[65636]> PkgRepo: verifying update for OPNsense
DBG(1)[65636]> PkgRepo: need forced update of OPNsense
DBG(1)[65636]> Pkgrepo, begin update of '/var/db/pkg/repo-OPNsense.sqlite'
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/meta.txz with opts "i"
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/meta.txz with opts "i"
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/meta.txz with opts "i"
pkg: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/meta.txz: No route to host
repository OPNsense has no meta file, using default settings
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/packagesite.txz with opts "i"
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/packagesite.txz with opts "i"
DBG(1)[65636]> Fetch: fetching from: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/packagesite.txz with opts "i"
pkg: https://pkg.opnsense.org/FreeBSD:11:amd64/20.1/latest/packagesite.txz: No route to host
Unable to update repository OPNsense
Error updating repositories!

Here the result of a ping to pkg.opnsense.org:

root@OPNsense02:~ # ping pkg.opnsense.org
PING pkg.opnsense.org (89.149.211.205): 56 data bytes
64 bytes from 89.149.211.205: icmp_seq=0 ttl=58 time=7.562 ms
64 bytes from 89.149.211.205: icmp_seq=1 ttl=58 time=7.574 ms

even when I use IPv4-Address for a https request for example i get this:

root@OPNsense02:~ # curl https://89.149.211.205/FreeBSD:11:amd64/20.1/latest/meta.txz                 
curl: (28) Failed to connect to 89.149.211.205 port 443: Operation timed out

So I think there must be something wrong on the TCP-Layer. - Layer 3 is working (Ping).

I would say a firewall-rule blocks the traffic, but the rules are the same on master and backup - synced.

When I do a TCP traceroute to the ip-address on the master I get this:

root@OPNsense01:~ # traceroute -P tcp 89.149.211.205
traceroute to 89.149.211.205 (89.149.211.205), 64 hops max, 40 byte packets
 1  utm320-1 (xxx)  0.420 ms  0.438 ms  0.214 ms
 2  xxx.de (xxx)  5.098 ms  5.345 ms  4.903 ms
 3  xxx (xxx)  4.464 ms  4.392 ms  4.536 ms
 4  et-1-0-0.bb04.ams-01.leaseweb.net (80.249.209.215)  7.459 ms  7.549 ms  9.662 ms
 5  be-104.br02.ams-01.nl.leaseweb.net (31.31.38.143)  11.371 ms  8.381 ms  7.948 ms
 6  be-11.cr08.ams-01.nl.leaseweb.net (81.17.35.185)  12.430 ms  12.247 ms  11.249 ms
 7  po-1.ce18.ams-01.nl.leaseweb.net (81.17.35.69)  9.807 ms
    po-1.ce17.ams-01.nl.leaseweb.net (81.17.35.67)  9.541 ms
    po-2.ce17.ams-01.nl.leaseweb.net (81.17.35.71)  9.530 ms

In the Firewall-Logs of the master I get passed packets for that trace

When I do that trace on the backup-appliance I get no logged packets on the firewall of the backup-appliance...

Btw FYI: I use CARP-Interfaces as well

[franco]

What if you use ping6 instead? As I said, maybe IPv6 is configured but not working properly.

In that case either fix IPv6 WAN config or disable in there.


Cheers,
Franco

[jimbeam128]

Hi Franco,

the interfaces were all disabled for ipv6.

I went on investigating this issue and could isolate it a little bit more - what I absolutley don´t understand:

Here my procedure:

- I cloned the Master-VM
- I reconfigured the master-vm and changed the ip-addresses an so on to be the backup-appliance...

-> communication to "internet" doesn´t work.

Then I changed the IP-Address of the interface from 192.168.xxx.171 to 192.168.xxx.172. The rest stays the same. After that change the connection to the internet works and update is possible. - When I change it back to 171 then the connection is broken again. - reproduceable...

There are no firewall-rules configured (by myself) which block traffic from 171.

I have a working workaround for my problem, but it´s not solved.

Are there maybe any rules which are not presented in the GUI?

[diskonekted]

Hi Jimbeam128,

I had a very similar problem today, but managed to get it resolved with some of the info here and making some config changes - perhaps what I did will help you or some others.

In short, I had to update the 'point-to-point' devices to select my WAN port as the PPPoE interface. This was done by editing the PPPoE interface under 'Interfaces > Point-to-point > Devices' and then changing the 'Link Interfaces' to that of both my 'WAN' and the physical adapter (re0, in my instance).

The longer version is that the setup wizard when i first commissioned the box had also assigned a IPv6 config (or perhaps I'd accidentally set it, can't recall). Either way I'd had both a IPv4 and IPv6 config on my WAN since first run and whilst the tunnel would establish - it was creating some issues with connectivity (OPNsense updating was one of those).

After finding this thread I thought I'll clear up my stale IPv6 config once and for all; but didn't follow the cardinal rule......take a backup first! So this lead me down a ~30 minute rabbit hole to try and diagnose why the PPPoE config in the 'Interfaces > WAN' page was correct, but not establishing. It wasn't until I tried the 'Advances and MLPPP' link that lead me through to 'Interfaces > Point-to-point > Devices' that I saw the PPPoE interface was only bound to /dev/cu2. A quick edit to that config and connectivity resolved immediately.

Returning to perform a package update - the mirror addresses resolved straight away and updates are flowing! (for note, migrating from v23.1 to v23.7.9).

Might not fix your issue - but thought I'd mention what worked for me.