Topics

Hi there,

I recently set up an appliance to be used as cold spare / backup for the case that the primary system dies:
- Installed OPNsense (24.7.x)
- Installed BEmanager
Everything working so far.

Then I exported the active BE (24.1.x) from the primary system to a network storage using BEmanager and then imported this BE to the new backup system. After activating this BE and rebooting, I get the following error and the device unable to boot:

Code Select Expand


Mounting from zfs:zroot/ROOT/restore-2024-10-24-061121 failed with error 45.
Loader variables:
vfs.root.mountfrom=zfs:zroot/ROOT/restore-2024-10-24-061121


I am pretty sure I've done everything the same way I did it several times before when creating backup devices or moving to new devices.

Primary device:

bectl list
BE        Active Mountpoint Space Created
24.1.10   NR     /          2.26G 2022-01-13 06:34
24.1.10_2 -      -          2.91M 2024-10-24 07:50

Where I exported the active 24.1.10 BE.
(I had to create 24.1.10_2 BE since BEmanager will not recognize any BE for export when only one BE exists; however, it is the same way I've done it several times before.)

Backup device after importing BE:

Code Select Expand


bectl list
BE                        Active Mountpoint Space Created
default                   NR     /          1.12G 2024-10-24 05:22
restore-2024-10-24-061121 -      -          2.26G 2024-10-24 06:11

root@OPNsense:~ # bectl activate restore-2024-10-24-061121
Successfully activated boot environment restore-2024-10-24-061121

root@OPNsense:~ # bectl list
BE                        Active Mountpoint Space Created
default                   N      /          1.12G 2024-10-24 05:22
restore-2024-10-24-061121 R      -          2.26G 2024-10-24 06:11

root@OPNsense:~ # reboot



I am not really familar with ZFS and BE, though I have no idea where to start debugging,
but I now found that the SSD of the backup system only has 8GB where the primary system has 30GB, though ZFS partition size is 5.2GB on backup system and 22GB on primary system. Also swap size differs.
Could this cause any problems? Any other ideas?

Cheers

Here are some screenshots and experiences with the new dashboard (90% browser zoom, Darkmode by DarkReader Firefox plugin):



I found that the Disk Usage widget shown as diagram will change to table mode when the widget is enlarged to at least two rows. Since I am not a fan of coloured diagrams I like this option.



Same for Firewall diagram, on first screenshot when it's size is one row it shows less information than on two rows...



... enlarging this widget to three rows it will also turn into table mode and firewall log. Nice!



My impressions and wishes:
For firewall widget it would be nice to separate live log from events when in table mode.
See screenshot 4, having longer v6 addresses, source and destination are overleaping in some cases, maybe it is possible to shorten them in those cases.
For CPU widget it would be nice to have the option to reduce it to total graph only.

After years working with the old dashboard, this new and modern dashboard is kinda strange for me and I am sure I will miss the old simple and plain one for the first time. What I like is the option to have different sizes for each widget.


Next I would like to test OVPN DCO, I just don't know when I'll have time to do it :(

Hi there,

I switched to Kea yesterday and everything is working fine so far, Kea distributes dynmaic leases and also configured static leases. Those leases are shown in leasing table for a while, but then they disappear and the list is empty. However, all devices are still working with their assigned IP.
Powering on another device, it will get it's IP and this lease is shown in the list, again for only a while and then disappears.
For the time being I don't know when leases disappear, but it feels this happens when I get those entries in log (lease file cleanup):

Code Select Expand

2024-02-09T11:01:28 Informational kea-dhcp4 INFO [kea-dhcp4.dhcpsrv.0x8346c0000] DHCPSRV_MEMFILE_LFC_EXECUTE executing Lease File Cleanup using: /usr/local/sbin/kea-lfc -4 -x /var/db/kea/kea-leases4.csv.2 -i /var/db/kea/kea-leases4.csv.1 -o /var/db/kea/kea-leases4.csv.output -f /var/db/kea/kea-leases4.csv.completed -p /var/db/kea/kea-leases4.csv.pid -c ignored-path
2024-02-09T11:01:28 Informational kea-dhcp4 INFO [kea-dhcp4.dhcpsrv.0x8346c0000] DHCPSRV_MEMFILE_LFC_START starting Lease File Cleanup

Known issue?

Cheers

[[PROLOGUE]]

[PROLOGUE]
Ever since I became a member of this forum, I keep reading about problems with cellular modems from Sierra Wireless (which doesn't mean that Sierra Wireless modems are bad).

In the meantime I have switched to a different, external modem because I also had to struggle with these problems for some time. Over the years, I created a small troubleshooting guide that I could use when nothing worked again. Now I don't need this help anymore, but I don't want to let it get lost in my documents either, but share my findings and make them available here, maybe it will help someone after all.
Please note that it has been a year since I last used my MC7304 before I replaced it with an external modem. This time OPNsense 21.x was present with hardened BSD.

[GENERAL]
Below I describe what I found out for my setup, which doesn't mean that it works in all cases.
Most of the problems occur because the OS cannot communicate with the modem or when connecting to the mobile network itself.

1) Check modem support!
BSD not really seems to support cellular modems very well, so you should always check if someone has already reported that the modem has been successfully set up in BSD or OPNsense. Also remember that there are different cellular bands used by different geographical regions: Your modem must be compatible with your region! MC7304 will perform fine in Europe/ Germany and Australia, but not really good in USA as the modem only supports 2100MHz for that region.

2) Be patient!
After initial connection attempt, e.g. after reboot, it may take some attempts over a few minutes until the connection will be established.

3) Read the logs!
The point to point connection logs are essential to debug such problems, but be prepared that there may be created tons of entries within a few seconds while the modem is trying to connect. You´re good to go using filters for some specific keywords I´ll refer to later in ,,Reading the logs" section.

4) Don´t worry!
Some PPP configuration settings such as service provider settings may dissapear in the GUI occasionally, that should not be the problem, in most cases the connection will work without these settings displayed in GUI.


As mentioned in the docs, the SIM PIN definitely should be deactivated. Do this with a cellular phone, or using the CLI if the access to the modem already works. See ,,AT-commands" section for further information.

Sierra Wireless cellular modems are regulary devices called ,,cuaU0.2", where ,,0" can also differ or may change occasionally for unknown reasons. "cuaU0.1" may be the GPS device included in most cellular modems.

The AT-command / init string &F0E1Q0 +CMEE=2 is often mentioned, which is supposed to prevent or fix problems. For me that AT-command never has fixed any issues and if I translated the individual commands correctly, this command may only provide additional debug information if the parameters differ from default. The command at&v will show you all current parameters.
&F1 will set all parameters to default, &F0 should have no effect, but I am not really sure.
E1 turns echo mode on, but it should be on by default.
Q0 turns result code presentation mode off, but it is by default.
+CMEE=2 reports mobile termination errors, again that's the default behaviour.

[AT-COMMANDS]
The docs states this way to locate the correct port while setting up the first time, but I recommend to do this step each time when issues occur, because it is (for me) the fastest way to see if any communication with the modem itself is possible:
Enter CLI / command prompt and enter cu -l /dev/cuaU0.2 to call the device, where ,,0.2" is your cellular device. If the output is ,,connected" the communication to the hardware is fine so far.
Now enter AT which should say ,,OK" and enter AT+CPIN. An ,,OK" states that access to SIM is also working. If ,,AT" says ,,all ports are busy" (ICP stage failure) you can do as follows:
Exit cu with ~ or start a new CLI session, then enter
cd ..
cd ..
rm var/spool/lock/LCK..cuaU0.2
where ,,0.2" is your device and reboot. Sometimes editing the interface worked form me without requiring a reboot.
If ,,AT+CPIN" gives an error you should check SIM card an PIN.
To disable SIM PIN, after calling your device, you can use AT+CLCK="SC",0,"xxxx" where xxxx is the current PIN.

For further troubleshooting you can use the following AT commands:
Reset modem (I experienced best results using this command in interface config as init string (without ,,AT")): AT!GRESET
Show operation status of the modem: AT!GSTATUS
Determine signal strength (2-9 marginal, 10-14 OK, 15-19 good, 20-30 excellent): AT+CSQ
Network registration status: ATt+CREG?
Show ISP: AT+COPS?
List supported transmission types in actual network: AT*CNTI=1
Show used transmission type: AT*CNTI=0

[READING THE LOGS]
Instead of starting troubleshooting in CLI, you can check the PtP logs in the GUI. I recommend to filter the output using the following keywords one by one:

,,Ack_Sent"
followed by ,,->opened" indicates that the internal hardware communication (ICP stage) is fine. If it shows an error / failure such as ,,The modem is not responding to ,,AT" ...", see ,,all ports are busy (ICP stage)" in AT-commands section further up.

,,MTU", ,,MRU", ,,ACCM"
followed by corresponding parameters indicates that parameter negotiation (LCP stage) was successful. I never experienced errors in this stage. If there are no parameters shown, the signal quality may be poor or modem parameters are misconfigured. It may be worth a try to reset parameters to factory defaults using AT&F1.

,,LCP"
followed by ,,authorization successful" indicates that the registration with the APN using username and password was successful (Authentication stage). If it shows an error / failure check your config.

,,IPCP"
followed by ,,state change Ack-Sent --> Opened" and an assigned IP address indicates that the cellular connection is established and ready to use (IPCP stage). If it shows no IP or an error / failure, check your PtP connection, select your country, ISP and all required options and save. This happened to me most times after a reboot, sometimes it took a while until the connection was established, sometimes it failed even after some hours of waiting. If ,,edit and save" don't do the trick, try a reset (AT!GRESET) in CLI or add !GRESET as init string in the interface configuration. Note that the init string may dissapear after some time. There is no need to add it again as long as everything works as expected.
I also remember that I had some issues at this stage (?) when I tried to get IPv6 to work on the cellular interface, maybe you're good to go disabling IPv6 temporarily.

[EPILOGUE]
I hope this little guide can help someone troubleshooting problems with Sierra Wireless modems. Maybe you can do something with it when using other modems, although my recent experiences show that there can be significant deviations here.
Please excuse my partly bad English, I read a lot in English, but that doesn't mean I can express myself better in that language, but what are online translators for?! ;)

Now while you guys solve your problems, I'll just sit back and watch my external modem at work... I'll have a drink to that, cheers!

Moin zusammen,

entsprechend diesem (https://forum.opnsense.org/index.php?topic=12379.0) älteren Thread versuche ich momentan sowohl mein globales v6 Prefix (tracked vom WAN) als auch ein ULA Prefix mittels radvd im LAN zu verteilen, allerdings wird stets nur die globale Adresse verteilt.

Config virtuelle IP:


Config radvd:


Wenn ich mir die radvd.conf anschaue siehe das so aus:

Code Select Expand


# Generated for DHCPv6 server lan
interface igb0 {
        AdvSendAdvert on;
        MinRtrAdvInterval 200;
        MaxRtrAdvInterval 600;
        AdvLinkMTU 1500;
        AdvDefaultPreference medium;
        AdvManagedFlag on;
        AdvOtherConfigFlag on;
        prefix 2a00:6020:xxxx:xxxx::/64 {
                DeprecatePrefix on;
                AdvOnLink on;
                AdvAutonomous on;
        };
        RDNSS fd00:10:13:12::1 {
        };
        DNSSL xxxxx {
        };
};


Müsste hier nicht auch entsprechend ein Eintrag für die ULA enthalten sein? Also etwo so

Code Select Expand

 
prefix fd00:10:13:12::/64 {
                AdvOnLink on;
                AdvAutonomous on;
        };

Aktuell bin ich auf Version 22.1, vorher habe ich noch nicht mit ULAs gearbeitet. radvd / die Sense selbst wurde mehrfach neu gestartet. Die globalen Adressen werden momentan mittels SLAAC und DHCP6 verteilt, wenn das mit der ULA klappt soll es aber nur noch mittels SLAAC laufen.

Ideen was hier verkehrt ist?

Gruß

Hi all,

I freshly installed RC1 on my "testing"-device whereby I set up everything (except os-wol and os-dyndns) by hand one-to-one according to my configuration of the production system (21.7).
Now I´m struggling with two issues that didn't occur with (almost)* identical configuration on 21.7.

* The main difference is that I used PPP LTE (internal Sierra Wireless modem, IPv4 only) as failover with 21.7, with 22.1 (meanwhile RC2) I´m using an external LTE modem DHCP4 (had issues using static IP) and DHCP6.

Issue #1:
I´m used to have a seperate OVPN server (hosted on OPNsense) for NAS devices, wherby one device is connected from OPNsense LAN side to the WAN side via global v6.
Since RC1 it is no more possible to connect from LAN side to WAN side (even with OVPN/WG on Android), no entries in FW log, OPNsense is directly attached to FTTH modem.

What is the reason for this behavior?
I´m just surprised that this is how it has worked so far, now my LAN side NAS is connected to VPN via link-local.

Issue #2:
Since RC1 (external) DNS resolution over VPN (OVPN and WG) is not working.
I found that DNS requests will be routed correctly to OPNsense and that the resolver (tried unbound, DNScryptProxy and AdGuard) receives and processes the request, but there will be no reply. Packet capture only shows requests from VPN client to Sense, I don´t know where the reply (if there is one at all) goes to, I can´t either find them on the other interfaces.

Maybe this is my configuration fault, I´m really not sure and surpised again, because identical config worked on 21.7 and previous versions...

Any ideas why the aspected behaviour is not happening or what I´ve done wrong?

Hi everyone,

on a freshly installed system DNScryptProxy is not working.
The service is starting fine but is not responding to DNS requests. All service logs in GUI are empty, even if all severeties are selected.

Log via CLI looks fine so far:

Code Select Expand

# vi /var/log/dnscrypt-proxy/dnscrypt-proxy.log
[2022-01-14 19:10:24] [NOTICE] dnscrypt-proxy 2.0.45
[2022-01-14 19:10:24] [NOTICE] Network connectivity detected
[2022-01-14 19:10:24] [NOTICE] Now listening to 127.0.0.1:5353 [UDP]
[2022-01-14 19:10:24] [NOTICE] Now listening to 127.0.0.1:5353 [TCP]
[2022-01-14 19:10:24] [NOTICE] Now listening to [::1]:5353 [UDP]
[2022-01-14 19:10:24] [NOTICE] Now listening to [::1]:5353 [TCP]
[2022-01-14 19:10:24] [NOTICE] Now listening to :53 [UDP]
[2022-01-14 19:10:24] [NOTICE] Now listening to :53 [TCP]
[2022-01-14 19:10:24] [NOTICE] Source [public-resolvers] loaded
[2022-01-14 19:10:24] [NOTICE] Loading the set of whitelisting rules from [whitelist.txt]
[2022-01-14 19:10:24] [NOTICE] Firefox workaround initialized
[2022-01-14 19:10:24] [NOTICE] Loading the set of blocking rules from [blacklist.txt]
[2022-01-14 19:10:24] [NOTICE] Loading the set of cloaking rules from [cloaking-rules.txt]
[2022-01-14 19:10:24] [NOTICE] Loading the set of forwarding rules from [forwarding-rules.txt]
[2022-01-14 19:10:27] [NOTICE] [dnscrypt.be] OK (DNSCrypt) - rtt: 21ms
[2022-01-14 19:10:29] [NOTICE] [dnscrypt.eu-nl] OK (DNSCrypt) - rtt: 17ms
[2022-01-14 19:10:29] [NOTICE] [quad9-doh-ip6-port443-filter-pri] OK (DoH) - rtt: 10ms
[2022-01-14 19:10:29] [NOTICE] [quad9-dnscrypt-ip4-filter-pri] OK (DNSCrypt) - rtt: 8ms
[2022-01-14 19:10:29] [NOTICE] [quad9-dnscrypt-ip4-filter-pri] OK (DNSCrypt) - rtt: 8ms - additional certificate
[2022-01-14 19:10:29] [NOTICE] [quad9-doh-ip4-port443-filter-ecs-pri] OK (DoH) - rtt: 13ms
[2022-01-14 19:10:32] [NOTICE] [quad9-doh-ip6-port5053-filter-pri] OK (DoH) - rtt: 17ms
[2022-01-14 19:10:32] [NOTICE] [dns.digitale-gesellschaft.ch] OK (DoH) - rtt: 19ms
[2022-01-14 19:10:32] [NOTICE] [dns.digitale-gesellschaft.ch-2] OK (DoH) - rtt: 19ms
[2022-01-14 19:10:32] [NOTICE] [dnscrypt.eu-nl-ipv6] TIMEOUT
[2022-01-14 19:10:32] [NOTICE] Sorted latencies:
[2022-01-14 19:10:32] [NOTICE] -     8ms quad9-dnscrypt-ip4-filter-pri
[2022-01-14 19:10:32] [NOTICE] -    10ms quad9-doh-ip6-port443-filter-pri
[2022-01-14 19:10:32] [NOTICE] -    13ms quad9-doh-ip4-port443-filter-ecs-pri
[2022-01-14 19:10:32] [NOTICE] -    17ms dnscrypt.eu-nl
[2022-01-14 19:10:32] [NOTICE] -    17ms quad9-doh-ip6-port5053-filter-pri
[2022-01-14 19:10:32] [NOTICE] -    19ms dns.digitale-gesellschaft.ch
[2022-01-14 19:10:32] [NOTICE] -    19ms dns.digitale-gesellschaft.ch-2
[2022-01-14 19:10:32] [NOTICE] -    21ms dnscrypt.be
[2022-01-14 19:10:32] [NOTICE] Server with the lowest initial latency: quad9-dnscrypt-ip4-filter-pri (rtt: 8ms)
[2022-01-14 19:10:32] [NOTICE] dnscrypt-proxy is ready - live servers: 8


Reinstalled service without success and tried again where I deleted all obviously related service-files via CLI; after reinstalling the service, all previous configs were restored.
=> how to completly remove all related service files?
May this be a config-fault? (Did the same config as on my 21.7 system; same hardware)

I'll be happy to send you any additional information you need...

Moin zusammen,

ich habe seit jeher das Problem, dass meine IPv6 Konnektivität bei Änderungen der Interface Einstellungen (egal ob neues VLAN oder kleine Anpassungen an LAN oder WAN) verloren gehen, und auch wenn die Sense hardwareseitig kurz vom LAN getrennt wird.
Dabei verliert zunächst das WAN Interface seine v6 und demzufolge auch alle anderen Interfaces, die sich daraus eine v6 ableiten.

Abhilfe schafft nur, wenn ich alle Dienste auf v6 stoppe (OpenVPN und Wireguard), auf allen nicht-WAN Interfaces v6 deaktiviere und anschließend die Einstellungen vom WAN Interface ohne Änderungen speichere oder die Sense neu boote. Beim Reaktivieren von v6 auf den Interfaces ist es manchmal ein Glückspiel, ob die v6 auf dem WAN erhalten bleibt oder wieder verloren geht.

Der Aufbau des Netzwerks ist eigentlich ganz simpel, VLAN sind mittlerweile deaktiviert:

Code Select Expand


                mPCIe    DHCP4
LTE-Modem        =>      [LTE] |---------------|
                               |   OPNsense    | [LAN]    =>    Switch
Glasfasermodem   =>      [WAN] |---------------|
                Eth.     DHCP6             Track Interface (WAN)
                         DHCP4                  static v4


Am WAN bekomme ich von der Deutschen Glasfaser ein /56 Präfix, das ich idR problemlos weiter delegieren kann, zumindest wenn ich denn erstmal eine v6 habe.
Das LTE Interface ist nur IPv4 Failover, also auch nur mit IPv4 konfiguriert und sollte denke ich keinen Einfluss auf die Problematik haben.
Firmware ist 21.7.6, das Verhalten habe ich aber schon seit ich v6 verwende, schätze seit 19.1.

Hat jemand eine Idee wie das Problem gelöst werden kann?
Vor allem die Tatsache, dass die v6 am WAN verloren geht, wenn der Link am LAN down ist, macht mir etwas zu schaffen...

Gruß und ein schickes Wochenende, wenn es so weit ist.

Moin zusammen,

ich habe NAT Portweiterleitungen für IPv4 und IPv6 eingerichtet, welche externe DNS Anfragen an die OPNsense/ DNScryptProxy umleiten.

Sowohl für die Port-Forward Regeln, als auch für die zugehörigen Firewall-Regeln habe ich das Logging aktiviert.
Die Port-Forwards werden sowohl für v4 als auch für v6 geloggt, die Firewall-Regeln hingegen werden nur für v6 geloggt, nicht aber für v4. Die Weiterleitung unter v4 funktioniert einwandfrei und DNScryptProxy beantwortet die Anfragen entsprechend.

Ich kann mir das Verhalten nicht erklären, übersehe ich hier etwas oder handelt es sich um einen Bug?

Gruß

Moin zusammen,

bevor es zur Sache geht so wie ich es vor Jahren mal gelernt habe kurz einmal zu mir:
Ich nutze opnsense seit vergangenem Jahr für mein Heimnetz, insbesondere wegen des OVPN Servers und WAN Failover. Bei der Einrichtung und sonstigen Problemen habe ich mich immer hauptsächlich durch die Docs und Forenbeiträge geschummelt oder gemäß Try and Error ausprobiert... demnach hält sich meine Erfahrung mit der gesamten Materie etwas in Grenzen, außerdem bin ich GUI-Liebhaber der nur hobbymäßig Bezug zur IT hat ;)

Beim Wechsel zu einem neuen ISP (Deutsche Glasfaser) im verganenen Monat habe ich auch meine Hardware gewechselt (Terra/ Wortmann RC100 G3) und opnsense 20.1.7 entsprechend neu aufgesetzt.
Mein ISP liefert mir CGNAT und natives v6, opnsense hängt direkt am Modem, alles läuft soweit einwandfrei.

Was mir auffällt sind jedoch Einträge in der FW-log auf die ich mir keinen Reim bilden kann, wobei igb0 und 10.13.12.0/24 mein LAN ist.

Code Select Expand

filterlog: 16,,,0,igb0,match,block,in,4,0x0,,64,34390,0,DF,6,tcp,76,10.13.12.60,172.217.16.138,46232,443,24,FPA,2298045867:2298045891,3739770163,376,,nop;nop;TS

Das gleiche tritt auch auf IPv6 und von unterschiedlichen Clients auf...
Wie kann es sein, dass hier Pakete geblockt werden, die vom LAN ins WAN gehen, obwohl eine default allow LAN to any Regel besteht?

Einschränkungen merke ich wie gesagt nicht, die Einträge sind nur ziemlich lästig und bereiten mir die Sorge, dass irgendwo doch der Wurm drin ist...

Sollten noch Infos benötigt werden stelle ich diese natürlich gern zur Verfügung, ich will jetzt nur nicht mit redundanten Infos um mich werfen...

Gruß