Messages

will do, thanks again  :)

Hi,

OK, it works now as expected:
- connectivity Test works
- ping from SourceAddress=Default works
- check for Updates works
- this weird route entry is gone

I did delete all Groups and Single Gateways.
Rebooted.
Deactivated in "System: Settings: General" the "Gateway switching" option
Created NordVPN as Single-Gateway again:
Interface=WAN
IP=192.168.179.254
Upstream=YES
DisableMonitor=YES
Priority=1

I reckon the upgrade process of my "weird" configuration messed up somehow (if that is possible).

Thanks for your help, highly appreciated! The topic is closed from my point of view.

If you like to investigate further I'm here. Or maybe I can send you a config-file etc. of my v22.1 so you can debug/test it yourself...

thanks
Soko

No... not that I know of.... The list @ "Interfaces: Virtual IPs: Settings" is empty.

"Disable Force Gateway" was checked already on my config (can't remember why I did that though)

Here is another interesting thing I've found and is different to v22.1

Code Select Expand


Proto    Destination         Gateway             Flags Use     MTU      Netif   Netif (name) 
ipv4     default             192.168.179.254     UGS   NaN     1500     em1     wan           
ipv4     127.0.0.1           link#4              UH    NaN     16384    lo0     Loopback           
ipv4     192.168.179.0/24    link#2              U     NaN     1500     em1     wan           
ipv4     192.168.179.253     link#2              UHS   NaN     16384    lo0     Loopback           
ipv4     192.168.179.254     link#2              UHS   NaN     1500     em1     wan           
ipv4     192.168.254.0/24    link#1              U     NaN     1500     em0     LAN           
ipv4     192.168.254.253     link#1              UHS   NaN     16384    lo0     Loopback     


The third line from the bottom does not exist on v22.1!
If I delete it, it still does not work. After a reboot after deleting it it is here again...

Thanks for your time!

your command outputs:
default             192.168.179.254   UGS   em1

EDIT: Thats the IP of the Gateway NordVPN.
WAN IP is 192.168.179.253/24

If I haven't mention it before: For the clients everything works! Its just seems that OPNsense itself has no internet connection in v22.7

[Interfaces: Diagnostics: Ping]

Hmmm...its seems that I'm not even able to ping NordVPN in with SourceAddress=default.
What exactly is this SourceAddress=default?

EDIT: Just tried on v22.1: Ping from SourceAddress=default works there! So I think the root problem is this.

As seen below pinging &traceroute NorVPN and pkg.opnsense.org works fine!?!

So how can this be an issue of my NordVPN gateway?

Interfaces: Diagnostics: Ping
Host: 89.149.211.205
Fails with SourceAddress: default
Succeeds with SourceAddress: LAN, WAN

Interfaces: Diagnostics: Ping
Host: 192.168.179.254 (NordVPN Gateway)
Fails with SourceAddress: default
Succeeds with SourceAddress: LAN, WAN

Interfaces: Diagnostics: Trace Route
Host: 89.149.211.205
Fails with SourceAddress: default
Succeeds with SourceAddress: LAN, WAN

Code Select Expand


# /usr/sbin/traceroute -w 2 -n  -m '18' -s '192.168.254.253'   '89.149.211.205'
traceroute to 89.149.211.205 (89.149.211.205) from 192.168.254.253, 18 hops max, 40 byte packets
1  192.168.179.254  0.397 ms  0.246 ms  0.249 ms
2  10.8.0.1  60.508 ms  25.678 ms  25.393 ms
3  37.120.155.225  28.335 ms  28.452 ms  33.759 ms
4  176.10.83.68  64.103 ms  105.169 ms  58.466 ms
5  82.102.29.36  29.961 ms
    82.102.29.30  28.370 ms
    82.102.29.36  25.943 ms
6  176.10.82.102  33.783 ms  37.875 ms  37.917 ms
7  89.44.212.109  52.182 ms
    89.44.212.5  47.829 ms  46.059 ms
8  77.243.176.217  46.043 ms  48.031 ms  43.809 ms
9  195.66.225.56  45.818 ms  44.217 ms  49.654 ms
10  31.31.34.20  48.640 ms
    31.31.34.22  45.396 ms  48.409 ms
11  31.31.38.83  47.555 ms  46.046 ms
    31.31.38.119  45.977 ms
12  81.17.35.183  45.335 ms
    81.17.32.77  44.718 ms
    81.17.35.179  49.768 ms
13  81.17.35.73  47.918 ms
    81.17.35.67  47.924 ms
    81.17.35.71  46.359 ms
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *


Yes it is:

System: Gateways: Single:
NordVPN (active)   WAN   IPv4   254 (upstream)   192.168.179.254   103.86.96.100   ~   ~   ~   Online   

There is another single gateway, but that one is forced offline.
both of them are in a group with NordVPN being Tier 1.

The default IPv4 route also points to 192.168.179.254

System: Routes: Log File: shows something weird though:
CLOG��

I've seen this a couple of times in the logs so far

Here are my firewall rules.
Nothing in floating or loopback

So,

Finally the update check finished:

Code Select Expand


***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 22.7_4 (amd64/OpenSSL) at Mon Aug  1 12:52:51 CEST 2022
Fetching changelog information, please wait... fetch: transfer timed out
Updating OPNsense repository catalogue...
pkg: Repository OPNsense has a wrong packagesite, need to re-create database
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.7/latest/meta.txz: Operation timed out
repository OPNsense has no meta file, using default settings
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.7/latest/packagesite.pkg: Operation timed out
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.7/latest/packagesite.txz: Operation timed out
Unable to update repository OPNsense
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***


The connectivity check says:

Code Select Expand


***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 22.7_4 (amd64/OpenSSL) at Mon Aug  1 13:49:14 CEST 2022
Checking connectivity for host: pkg.opnsense.org -> 89.149.211.205
PING 89.149.211.205 (89.149.211.205): 1500 data bytes

--- 89.149.211.205 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:13:amd64/22.7
Updating OPNsense repository catalogue...


Which is kinda weird as an "Interfaces: Diagnostics: Ping" with SourceAddress=default fails...
but with SourceAddress=WAN succeeds. LAN too! Also success from the clients.

"Interfaces: Diagnostics: DNS Lookup" also succeeds of course.

Why is something suddendly a problem in v22.7... weird...

OK,

Now I've ungraded to v22.7 again.

First of all I had to disable dnsmasq and enable unbound to make the internet work again (nothing else be untick and tick the enable-checkboxes).

Then I goto System:Firmware:Status
This takes 1 minute until I see something here. In v22.1 it took 2 seconds. Here's the output:

Code Select Expand


Type opnsense
Version 22.7_4
Architecture amd64
Flavour OpenSSL
Commit 909dcabd5
Mirror https://pkg.opnsense.org/FreeBSD:13:amd64/22.7
Repositories OPNsense
Updated on Mon Aug 1 12:44:24 CEST 2022
Checked on N/A


Then I click "Check for Updates" which is stuck here:

Code Select Expand


***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 22.7_4 (amd64/OpenSSL) at Mon Aug  1 12:52:51 CEST 2022
Fetching changelog information, please wait... fetch: transfer timed out
Updating OPNsense repository catalogue...
pkg: Repository OPNsense has a wrong packagesite, need to re-create database


fyi: the "timed out" for fetching changelog info comes immediately.
there is no load on the cpu... so I don't think it is doing a re-create.

I've found: https://forum.opnsense.org/index.php?topic=7742.0
# /usr/bin/time configctl firmware check says:
OK  0.05 real    0.04user   0.00sys

While typing (after 5 mins or so) the messages continued:

Code Select Expand


***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 22.7_4 (amd64/OpenSSL) at Mon Aug  1 12:52:51 CEST 2022
Fetching changelog information, please wait... fetch: transfer timed out
Updating OPNsense repository catalogue...
pkg: Repository OPNsense has a wrong packagesite, need to re-create database
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.7/latest/meta.txz: Operation timed out
repository OPNsense has no meta file, using default settings


thx
Soko

Ahh OK!
So everything after the line "Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:4f00:a005:5::" is IPv6 stuff. Then yes... it can be discarded.

This was just my attempt to figuring out the issues I have once I'm on v22.7. (see opening post). The "Check for Update" there gets also stuck and does nothing (same as the Health Check).

I reckon I should open a separate Post in the 22.7 folder, right?

Yeah... I have disabled IPv6 "globally" (according to https://www.thomas-krenn.com/de/wiki/OPNsense_IPv6_deaktivieren) back in the days when I first installed OPNsense as I don't have or need IPv6.

Is missing IPv6 also causing the troubles in v22.7?

Is it enough if I set "IPv6 Configuration Type" on WAN back to DHCPv6 or do I have to "Allow IPv6" in Advanced Settings and redo the IPv6 rules in the Firewall?

thx heaps!

PS: With IPv6 to DHCPv6 the connectivity check says "No route to host" :(

Hi guys,

While trying to identify the issues I'm having with v22.7 since the last couple of hours I've found some strange message back when my VM is on v22.1.10_4:

Health Check is sound:

Code Select Expand


***GOT REQUEST TO AUDIT HEALTH***
Currently running OPNsense 22.1.10_4 (amd64/OpenSSL) at Mon Aug  1 10:35:02 CEST 2022
>>> Check installed kernel version
Version 22.1.9 is correct.
>>> Check for missing or altered kernel files
No problems detected.
>>> Check installed base version
Version 22.1.9 is correct.
>>> Check for missing or altered base files
No problems detected.
>>> Check installed repositories
OPNsense
>>> Check installed plugins
No plugins found.
>>> Check locked packages
No locks found.
>>> Check for missing package dependencies
Checking all packages: .......... done
>>> Check for missing or altered package files
Checking all packages: .......... done
>>> Check for core packages consistency
Core package "opnsense" has 66 dependencies to check.
Checking packages: .................................................................... done
***DONE***


Connectivity Check reports "Non-recoverable resolver failure"

Code Select Expand


***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 22.1.10_4 (amd64/OpenSSL) at Mon Aug  1 10:36:10 CEST 2022
Checking connectivity for host: pkg.opnsense.org -> 89.149.211.205
PING 89.149.211.205 (89.149.211.205): 1500 data bytes
1508 bytes from 89.149.211.205: icmp_seq=0 ttl=53 time=188.715 ms
1508 bytes from 89.149.211.205: icmp_seq=1 ttl=53 time=51.155 ms
1508 bytes from 89.149.211.205: icmp_seq=2 ttl=53 time=69.108 ms
1508 bytes from 89.149.211.205: icmp_seq=3 ttl=53 time=68.161 ms

--- 89.149.211.205 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 51.155/94.285/188.715/54.985 ms
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:13:amd64/22.1
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 799 packages processed.
All repositories are up to date.
Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:4f00:a005:5::
ping6: UDP connect: No route to host
Checking connectivity for repository (IPv6): https://pkg.opnsense.org/FreeBSD:13:amd64/22.1
Updating OPNsense repository catalogue...
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.1/latest/meta.txz: Non-recoverable resolver failure
repository OPNsense has no meta file, using default settings
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.1/latest/packagesite.pkg: Non-recoverable resolver failure
pkg: https://pkg.opnsense.org/FreeBSD:13:amd64/22.1/latest/packagesite.txz: Non-recoverable resolver failure
Unable to update repository OPNsense
Error updating repositories!
***DONE***


Which I quite not understand as the ping to pkg.opsense.org is OK and I can download https://pkg.opnsense.org/FreeBSD:13:amd64/22.1/latest/meta.txz from my clients with no problem.

The upgrade to v22.7 works. But afterwards I have several issues there:

• "Dnsmasq DNS" does not work anymore (clients don't get DNS resolved). I have to disable it and enable unbound
• /ui/core/firmware#status page does take 5 minutes to show values after reboot.
• Health Audit is stuck @ "Core package "opnsense" has 63 dependencies to check." at 1 dot of progress for 30 mins.

Are the issues on v22.7 caused by the "Non-recoverable resolver failure" in v22.1?

thx
Soko

[So I've tried the following config (IPv6 is generally disabled):]

Hi,

I'm trying to wrap my head around that issue and I think all this should work with no Gateway Group at all...

So I've tried the following config (IPv6 is generally disabled):

System: Gateways: Single:

• WAN_GW:  Prio=254 Upstream=true GW=192.168.179.254 MonitorIP=103.086.096.100
• FAILGW:  Prio=255 Upstream=true GW=192.168.179.001 MonitorIP=046.182.019.048

Usually there is (active) written behind WAN_GW

System: Settings: General:

• The monitor IPs of above are the DNS Servers with the according use gateway of above
• Allow default gateway switching = true

Interfaces: WAN:

• IPv4 Upstream Gatway = Auto-detect

System: Routes: Status:

• Destination=default Gateway=192.168.179.254
• Followed by two more entries for the monitor/DNS IPs as Destination with the corresponding Gateway

Firewall: Rules: LAN:

• The Default allow LAN to any rule has nothing selected as Gateway set

Firewall: Settings: Advanced:

• Sticky connections = false
• Shared forwarding = false
• Disable force gateway = true (Why? see below)

The test:

Now I shut down my WAN_GW (device with 192.168.179.254).

After a little wait I have the following:

System: Gateways: Single:

• WAN_GW Status=offline
• FAILGW Status=online and the (active) is now written behind this Gateway

System: Routes: Status:

• Destination=default Gateway=192.168.179.1
• Followed by two more entries for the monitor/DNS IPs as Destination with the corresponding Gateway

So everything should work => but it doesn't. I have no internet connection.

What doesn't help

• Setting FAILGW as Gateway for the Default allow LAN to any rule
• Disable force gateway = false: The auto-floating-rule created when this is false has still WAN_GW as Gateway. Even when it's offline.

What does help

• IPv4 Upstream Gatway = FAILGW for under Interfaces: WAN:

Conclusion
I my knowledge of networks I don't get why the above test is not working even when:

• The WAN interface has Auto-detect as GW
• No rule has a Gateway set
• The routing table has the correct default route to 192.168.179.1

Maybe someone can shed a light on this...
Or maybe a Multi-GW + Single WAN config has to be completely different to work.

thx
Soko

...