Messages

1

23.1 Legacy Series / Monit RootFs service stopped working: "unable to read filesystem '/' state"

« on: March 29, 2023, 09:42:02 pm »

Since about release 23.1.2 I have been getting complaints from the Monit service about the RootFs service not working.  The e-mail is as follows:

Code: [Select]

Does not exist Service RootFs

Date:        Wed, 29 Mar 2023 17:07:23
Action:      restart
Host:        my.opnsense.host
Description: unable to read filesystem '/' state

Your faithful employee,
Monit

This still doesn't work as of the recent 23.1.5 update.

I am running OPNsense 23.1.5 on a ZFS-based install.  The install was bootstrapped from a FreeBSD install via opnsense-bootstrap.  It does have a ZFS file system mounted on /, so I'm not sure how to interpret the "unable to read filesystem '/' state" message.

Is anyone else experiencing this?  If so, is there a fix?  For now, I have disabled the service check so I don't get spammed with Monit e-mails about this service failing.  I would like to have the check working, though.

2

23.1 Legacy Series / Re: LAGG slow/fails to configure since upgrading to 23.1

« on: March 29, 2023, 09:29:26 pm »

That could be it, but 23.1.5 would address that. Previously we recommended removing these tunables or moving them to /boot/loader.conf.local where they are not being triggered after bootup.

I checked the LAG configuration in my switch and note that everything checks out on both ends.  All the ports are set to Long timeout on the switch and I have Fast timeout unchecked.  Also, the default for Administrative Flow Control is Disable in the switch, which actually matches the tunable setting of dev.em.X.fc being 0 for the LAGG members on the OPNsense side.  I've changed the switch to Auto Negotiation and removed the tunables and will see if that helps matters when I do a test reboot of OPNsense later today.

Unless something else changed between 22.7.x and 23.1.x, I can't think what might have caused the OPNsense LAGG to fail to come up after a reboot.  None of the workarounds mentioned in the thread work for me.  The only thing seems to be rebooting my managed switch.

3

23.1 Legacy Series / Re: LAGG slow/fails to configure since upgrading to 23.1

« on: March 29, 2023, 08:32:59 pm »

Just updated to 23.1.5.   LAG stays up after reboot.

See the information from reddit. 

https://www.reddit.com/r/opnsense/comments/1255xr8/2314_lagg_wont_come_up_after_reboot/

The Reddit link is very useful, thanks.

In my case, updating to 23.1.5 today did not fix the problem of the LAGG coming up: I still had to reboot my switch.  But, one of the Reddit replies says, "It seems to be a driver issue and custom eee/fc tunables set for your NIC," and I have dev.em.0.fc set to 0 for my NICs.  Maybe that is the cause of the problem?  I will test and report back.

4

23.1 Legacy Series / Re: LAGG slow/fails to configure since upgrading to 23.1

« on: March 12, 2023, 09:31:12 pm »

BTW, this is still happening for me with the latest 23.1.3 update.  Rebooting my managed switch is the easiest way for the LAGG to be established at the OPNsense end, otherwise the LAGG does not come up properly at the OPNsense end. 

5

23.1 Legacy Series / LAGG slow/fails to configure since upgrading to 23.1

« on: February 24, 2023, 10:24:36 pm »

I have a 3-port LACP LAGG configured on my OPNsense system that is connected to a Cisco SG350 managed switch.  This has worked fine in previous versions of OPNsense going back years but since upgrading to 23.1 it gives problems.  Specifically, it has trouble becoming active (configured) after boot.  The individual laggports will change in status, with the flags moving through various states such as <>, <COLLECTING>, <ACTIVE, COLLECTING>, and even with some (but not all) in the desired <ACTIVE, COLLECTING, DISTRIBUTING> state.

This is what it looks like when it is properly configured:

Code: [Select]

$ ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4812098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP>
ether 00:eb:ca:c0:05:c5
laggproto lacp lagghash l2,l3,l4
laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: em3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

It seems that even after 10 minutes or so that the LAGG is still cycling through various states with the member laggports and the interfaces built on this LAGG going UP and DOWN accordingly as it tries to configure.  The easiest way to fix it is to restart the Cisco switch. 

Has the way LAGG interfaces are configured changed in 23.1?  I see these two entries in the Changelog for 23.1:

• interfaces: register LAGG, PPP, VLAN and wireless devices as plugins
• src: assorted FreeBSD 13 stable fixes for e.g. bpf, bridge, bsdinstall ifconfig, iflib, ipfw, ipsec, lagg, netmap, pf, route and vlan components

I don't understand the import of either of those statements.  This setup worked flawlessly up to 22.7.11 and so whatever the problem is now appears to have crept in with 23.1.

Any hints or suggestions on how to get the LAGG to activate reliably are most appreciated.

6

23.1 Legacy Series / Re: Unbound recommended configuration for reverse lookup for private IPv4 addresses?

« on: February 24, 2023, 09:27:55 pm »

I don't know whether it's a "best practice" or not but in our setup we use explicit Unbound domain overrides to do forward and reverse lookups for IPv4 private addresses not handled explicitly by the local Unbound.

We have two sites joined via a site-to-site IPSec VPN.  Each site has local (non-overlapping) IPv4 subnets and a local domain name for the addresses its Unbound manages.  In the Domain Overrides for each site, there is an N.N.N.in-addr.arpa override that sends the queries to the other site's Unbound for PTR (reverse) lookups, as well as a site.local.domain. entry that forwards queries for forward lookups.  It's done vice-versa at the other site.

I guess you could use a similar approach to forward all the local IPv4 ranges you're interested in to the outer network's DNS servers.

7

23.1 Legacy Series / Re: NUT (UPS) daemon is unable to start after upgrade

« on: January 29, 2023, 09:12:23 pm »

Does this do the job? https://github.com/opnsense/plugins/commit/16cbe99ebf

# opnsense-patch -c plugins 16cbe99ebf


Cheers,
Franco

This patch fixes the problem for me and allows NUT to start up.

8

General Discussion / Re: Periodic.conf tunables?

« on: November 19, 2022, 05:30:04 am »

Thank you.  I added the setting to /etc/periodic.conf.local.  I just updated to 22.7.8 and the file persisted across that update.

9

General Discussion / Periodic.conf tunables?

« on: November 10, 2022, 05:09:05 pm »

The official OPNsense documentation about tunables (https://docs.opnsense.org/manual/settingsmenu.html) says they are for loader.conf and sysctl.conf tunables. I have a ZFS setup on which I would like to enable a periodic scrub. FreeBSD has a built-in /etc/periodic/daily/800.scrub-zfs task that is disabled by default. I'd like to enable this via the daily_scrub_zfs_enable setting.

Assuming this can't be added as a tunable to the "System: Settings: Tunables" section, the normal way in FreeBSD would be to add it to /etc/periodic.conf or /etc/periodic.conf.local. Will the latter persist across updates?

(Are there any plans to add periodic settings to "System: Settings: Tunables"?)

10

Virtual private networks / Re: Split DNS for IKEv2 IPSec VPN on macOS

« on: May 04, 2022, 05:09:26 pm »

PS: If anyone has any insight on how to get DNS tools like host and dig to work from the client side I'd be glad to hear about it.

11

Virtual private networks / Re: Split DNS for IKEv2 IPSec VPN on macOS

« on: May 04, 2022, 05:07:56 pm »

Here is an update on this from me:

Well, it appears that split-DNS was actually "largely working" for me with the macOS IKEv2 built-in client.  It was the way I was testing it that made it seem like it wasn't working at all.

"Largely working" means that resolver-based client DNS resolution works.  More simply, hostname resolution works for commands such as ssh, ping, curl, etc.  Where DNS resolution fails is for tools such as host and dig. These use the wrong resolver at the client side.  (I had been testing with host and dig.)

Although it would be nice for everything to work, I can live with the "largely working" state right now. 

One thing that I did actually have to do to get split-DNS (or any IPSec VPN DNS) working is to add the IPSec client network range as an explicit access list in Services -> Unbound DNS -> Access Lists.  I believe this is because IPSec is not available as an interface to select in Services -> Unbound DNS -> General -> Network Interfaces, and so doesn't get included in the "Internal" access lists.  Without this explicit access list entry, I was getting REFUSED responses to DNS lookups from the VPN client to the server.

12

22.1 Legacy Series / Re: Duplicate Entries for Static DHCPv4 leases

« on: April 29, 2022, 06:09:49 pm »

In my case, the IOT device pulled a DHCP address from the pool.  I clicked on the + sign next to that lease entry and made that same address static.

The system does allow this.  It was my understanding this essentially creates that static reservation.  This way that client will ALWAYS pull the same IP from the pool.

I see folks testing this by creating a new static IP outside of the pool.  In my case, I'm looking to create a DHCP static reservation by using the + side in the leases tab.

I recall this happening to me recently, too.  Converting an existing lease to static via the "+" button (even when changing the IP in the form for the static entry) yielded duplicates for that MAC address, whereas creating a static DHCP lease from scratch (without the client having a pre-existing lease) does not.

The duplicates went away.  I don't remember whether it's because I nuked the leases or whether they just went away when the initial dynamic lease expired.

(Caveat: this happened at a time when I was converting my setup over from HA dynamic DHCP to static DHCP due to DNS registration issues.)

13

Virtual private networks / Split DNS for IKEv2 IPSec VPN on macOS

« on: April 26, 2022, 04:07:37 pm »

I have a "road warrior" IPSec IKEv2 VPN setup that is working for me, at least when it comes to split-tunnelling. I have been trying to get it to work with a split-DNS configuration so that VPN clients only use the VPN-provided DNS servers for the local VPN DNS domain and all other DNS requests (for domains other than that) should use the client's default DNS resolver.  It's the split-DNS setup that isn't working for me.

Can anyone confirm whether or not they've got this working with the built-in IKEv2 client in macOS Big Sur or newer?  If so, what was the magic needed to get this working?

I use Apple Configurator to create IKEv2 VPN profiles for macOS, so I don't mind if the solution involves that.

14

High availability / Re: Unbound DNS registration not working for failover DHCP configuration

« on: April 12, 2022, 04:27:48 pm »

Yes, the DHCP synchronisation protocol does not include the host name.


Cheers,
Franco

Many thanks, Franco for confirming this.

So, would it be fair to say if I wanted DNS registration of DHCP hostnames in a HA OPNsense setup I should opt for one of these options:

• Stick to DHCP static mappings
• Use DHCP relay or otherwise offload DHCP onto some external HA DHCP service

Is there any other solution I might have missed?  (I'm inclined to opt for DHCP static mappings.)

15

High availability / Unbound DNS registration not working for failover DHCP configuration

« on: April 11, 2022, 09:47:44 pm »

I have a HA setup currently running OPNsense 22.1.5.  The HA has been working well but recently, when I went to configure HA DHCP (https://docs.opnsense.org/manual/how-tos/carp.html#optional-setup-dhcp-server), I find that the resultant setup does not work correctly for me---at least it does not work as I expect it to do.

The way in which it is not working is this: I have the "DHCP Registration" option of Unbound enabled but hostnames are not registered correctly in the local DNS.  Hostnames are registered only in the DNS of the HA node whose DHCP server issued the DHCP lease of the client.

Is this how it is supposed to work?  As I say above, this is not how I would expect things to work.  I would expect DHCP-registered hosts to be resolvable in both the active and passive nodes.

Here is an example:

HA Node A leases include this:

Code: [Select]

Interface   IP address      MAC address       Hostname Description Start                   End                     Status Lease type
PLUMBING    192.168.115.174 00:50:56:97:38:32                      2022/04/11 18:29:56 UTC 2022/04/11 20:29:56 UTC |||||  active
APP         192.168.116.177 00:50:56:97:89:1b rum-dev              2022/04/11 18:32:02 UTC 2022/04/11 20:32:02 UTC |||||  active
HA Node B leases include this:

Code: [Select]

Interface   IP address      MAC address       Hostname Description Start                   End                     Status Lease type
PLUMBING    192.168.115.174 00:50:56:97:38:32 awx-test             2022/04/11 18:29:56 UTC 2022/04/11 20:29:56 UTC |||||  active
APP         192.168.116.177 00:50:56:97:89:1b                      2022/04/11 18:32:02 UTC 2022/04/11 20:32:02 UTC *      active
||||| = "Online" graphic; * = "Offline" graphic.


From HA Node A, I can resolve rum-dev but not awx-test and vice versa from HA Node B. I would expect that the "DHCP Registration" Unbound option would allow DHCP hostnames to be resolvable from both Node A and Node B.

In my DHCPv4 configuration I have the CARP VIP set for "DNS Servers" and "Gateway" and the "Failover IP" points to the real IP address of the opposite node in the HA pair.

Should this work or am I misconfiguring something?

Note: Static DHCP entries work fine, DNS-wise.