Messages

Wizards (for both firewall and outbound nat) worked great, I just dumped the CSVs and then imported and then cleared the old rules.

I have some fairly complicated stuff going on (dual-wan using gateway groups for failover, but with specific source IPs forced out certain gateways for monitoring) and it all seems to have moved over just fine. Upgraded to 26.7.1 without issue as well. :)

Going to have to tackle the DHCPd migration at some point next, thanks for the pointer dseven

OK, so even though I can see what appear to be the "correct" rules in the (new) section, those are just "representations" of the old ones.

It was unexpected (to me) that they would be displayed at all in (new) until after I have imported them.

I have no strong attachment to the old system (although I've been using it for years). I'll run through the wizard :)

Probably more pressing is that I need to get off of ISC DHCPd; there's no simple migration path to get reservations out of there and over to dnsmasq, is there?

I'm on 26.1.11_10

I checked out the Rules (new) section and all of my rules from the legacy section appear to have been added over there already. I don't recall doing any export -> import dance... but searching this forum I'm seeing references that the migration "wizard" is 100% manual.

Has this changed and things have now been migrated automatically? Or is the only way to see things in "Rules (new)" from a manual import? (it's possible I did this late one night then forgot... nobody else logs in to this OPNsense system but me though.)

Total noob here, just bought my first domain on porkbun and trying to get this setup. I get the same "No applicable existing records" error.

When you say you created an A record with the wrong IP, is that what you put in the Answer/Value box? What is supposed to go in there? Or are we just filling it in to finish the record and then ddclient fixes it?

You would need to create an A-record in Porkbun's DNS management system. Just set the IP address to 127.0.0.1, or any other "fake" IP (a private IP would potentially work as well). Once the record exists, then the API call through ddclient should be able to update the IP in the A-record to your public one.

I am seeing "weird" stuff in dhclient.leases for at least one of my WAN interfaces as well. It's AT&T Fiber, and I'm bypassing their "BGW" device so I'm getting DHCP directly from their network (there's no "IP Passthrough" happening here). I'm not on v25.x yet, either - this is on 24.7.12, so I don't think this is a recent change.

My leases file has 11 separate "lease { }" stanzas in it, each of them with different renew, rebind, and expire dates and times. My dates and times appear to be relatively correct, but it seems to have a bunch of leases expiring several cycles "in the future," which I don't understand.

For example, it's currently 18:53. I have leases with a renew at 18:56 (this makes sense), but then also 19:26, 19:56, 20:26, 20:56, 21:26, 21:56, etc, all the way up to 23:56. I don't get why I have "future" leases, since I didn't think DHCP worked that way? The lease time is 3600 seconds (1hr), and I know that default behavior is to renew at the midpoint... I just don't get why I have "lease { }" entries for leases that shouldn't have been issued yet.

I don't seem to have any actual problems here, FWIW, just mentioning what seemed like odd behavior in my leases DB file.

(I was just looking through the file because I wanted to know the lease time, because I want to change my WAN MAC and was wondering how long I'd be stuck without service, because AT&T gives each customer an IP that doesn't really change, and will just ignore requests until the previous lease expires.)

[best]

I think subject says most of it.

I have an ONT/ONU device on one of my WAN interfaces that has a management IP, but after a while it stops answering ARP requests for that IP (I suspect this might be a "security" feature but I'm not positive, but I don't think it's something I can easily change).

Easy fix is to just add a static ARP entry on my WAN interface, and an IP Alias to that interface in the same subnet, and I can have permanent management access to the device.

What is the best way to add a static ARP entry that will survive reboots as well as upgrades? There doesn't seem to be a way to do it in the GUI. I found this ~4 year old post that suggested adding some lines to /etc/rc.conf but I assume this won't survive an upgrade and isn't really the "right" way to go.

If it was on the LAN side, I know I could add a DHCPv4 static entry with the "ARP Table Static Entry" option enabled, but I don't know if this will work for something in a different subnet that isn't actually being served by the DHCPv4 daemon...

[only]

This was happening on 22.1.8 and is still happening after upgrading to 22.1.9. Unfortunately I'm not exactly sure when it started happening. I noticed it after I changed the router's hostname (while still running 22.1.8 ).

I have dual-WAN, plus a LAN interface, and an OPT/secondary LAN that is disabled most of the time (but does have a static IP).

I'm using a subdomain of a public domain (that I own) for my LAN, and my opnsense machine is named "router.bh.example.net"  (example.net is standing in for my public domain here).

If I use dig against the router's hostname from my LAN, it is returning not only the LAN IP, but also the OPT, and both WAN IPs.

This is obviously somewhat of a problem because then hosts on the LAN aren't sure which IP to try to access for HTTPS / ping / etc.

I'm able to "fix" this by changing Services -> Unbound DNS -> General -> "Network Interfaces" to only list my LAN interface... but this is not desirable if I should ever want to use the OPT/secondary LAN. I don't want both interfaces to have the same hostname, and I definitely don't want public WAN IPs being resolved (internally) to the LAN hostname of the system.

I'm fairly sure Unbound / OPNsense didn't have this behavior in the past, as I think I would've run into problems trying to ping router.bh.example.net otherwise. Anybody else noticing this?

This has been fixed.  See https://forum.opnsense.org/index.php?topic=28695.0 for information on how to configure USB quirks to make this work.

As the person who figured things out on that other thread... I wouldn't call this "fixed," more like "worked around the problem." I don't know enough about the internals of FBSD and Nut to fully understand why this worked correctly in the past and doesn't now. It would be much better for someone with that knowledge to fix it correctly, instead of requiring a system tunable / tweak to get the kernel to step out of the way.

OPNsense tunables do end up in loader.conf.local and are activated during kernel initialisation.

I appreciate the clarification, I wish this was documented more clearly in the official docs. I knew that "tunables" were just a way to set sysctl flags, but it wasn't obvious to me what the load order is. I assumed (incorrectly) that the process by which they were loaded happened after initial kernel load...

FWIW, I ended up moving to a "netclient" Nut config - I was passing through the USB device (running OPNsense virtualized on Proxmox VE). I just setup Nut on the PVE server directly and am connecting the OPNsense VM to it via Nut's "netclient" mode now instead.

It's probably all redundant in my case, since Proxmox should gracefully shutdown the OPNsense guest anyway... but I'd rather the guest shut itself down on its own terms.

[confirm]

Conversing with myself, but this felt like it should be a new post.

https://www.freebsd.org/cgi/man.cgi?query=usb_quirk&sektion=4&apropos=0&manpath=FreeBSD+13.1-RELEASE+and+Ports

You'll need to use usbcontrol to find your device's descriptor info, first.

Code Select Expand

usbconfig -d ugen0.2 dump_device_desc
ugen0.2: <American Power Conversion Smart-UPS 1500 FW:UPS 09.4 / ID18> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0200
  bDeviceClass = 0x0000  <Probed by interface class>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0000
  bMaxPacketSize0 = 0x0040
  idVendor = 0x051d
  idProduct = 0x0003
  bcdDevice = 0x0106
  iManufacturer = 0x0001  <American Power Conversion >
  iProduct = 0x0002  <Smart-UPS 1500 FW:UPS 09.4 / ID=18>
  iSerialNumber = 0x0003  <xxxxx redacted  >
  bNumConfigurations = 0x0001

Note the "idVendor" and "idProduct" as you'll need these.

in /boot/loader.conf.local, you'll need to add, where x is a number starting at 0 (you may have other quirsk already)
hw.usb.quirk.x = "vendorID productID minprodVer maxprodver QUIRKNAME"

MinprodVer / maxProdVer are only important if you have multiple UPSes attached and are trying to be specific... otherwise, they can be "0" and "0xffff" for min and max (should catch all versions).

In our case, the problem is that the kernel is attaching the USB HID driver to the UPS, which Nut is having problems detaching. There's a "quirk" to tell the HID driver to ignore it... UQ_HID_IGNORE

Note that normally OPNsense would want us to set this "hw.usb.quirk.x" value in the GUI as a "System tunable," but I suspect that doing this won't work because tunables aren't loaded until after the kernel and modules are situated... which is too late. Option B would be to hack one of the nut startup scripts and call usbconfig and tell it to detach the kernel driver from the device (as I discovered in my earlier post)... but I like just telling the kernel to gtfo, especially since loader.conf.local should survive reboots and upgrades.

So, in my example, the final line added to /boot/loader.conf.local (which didn't exist on my system, so I created it) should be:
hw.usb.quirk.0 = "0x051d 0x0003 0 0xffff UQ_HID_IGNORE"

I can confirm that this has worked around my problem with nut + usbhid-ups (an APC) on 22.1.8_1, after a reboot. Nut loads correctly at boot and everything is working.

How do we fix this in OPNsense "correctly" instead of hacking around it? I dunno... maybe have nut run as root again instead of the uucp user? (I'm not sure why it's running as uucp in the first place?) Alternatively, figure out how to add the correct kernel-level permission to allow nut's user account to detach the kernel device driver without being UID 0. But my little work-around here is functional... just a pain in the butt since it requires manual intervention and is hard to script.  :P

[as root]

I've been having problems with this since 22.1.7, possibly a little earlier...

My first question / concern - is nut supposed to be running as the "uucp" user?

What I'm noticing, using a usbhid-ups driver, is that it chokes on access to a socket file that is supposed to live in /var/db/nut.

If I manually start that driver as root (/usr/local/libexec/nut/usbhid-ups -a <upsname> -DD -u root), it throws a complaint that the permissions on this socket file are wrong and that it has fixed them.

From that point forward, I can kill the debug copy, and then "restart" nut in the GUI and it works OK... but this doesn't survive a reboot.

So then I tried rm'ing the entire contents of /var/db/nut, and then restarting again... and this seems to work OK, but I suspect it too won't survive a reboot.

I've seen discussion elsewhere that the problem is permissions on the /dev tree (and specifically the USB entry for the UPS), but that did not need to be modified on my system.

I'm trying to figure out now how to permanently correct the permission issue so it survives a reboot...

[edit]
OK, it's not the /var/db/nut thing, permissions are fine by default.

But trying to run the usbhid-ups "nut driver" as root first is key. That forces the kernel USB HID driver to detach from the device, which then allows nut to bind to it, and everything works OK. Then you can kill off that process, get the GUI to start the driver as normal, and it all works.

Work-around:

Code Select Expand

usbconfig -d <usb device entry - mine is ugen8.2> detach_kernel_driver

After running that, you can kick the nut service in the webUI and everything works.
So, this is something weird going on with FBSD13 and the HID driver not wanting to let go of the device when nut asks for it; presumably somehow combined with permissions (maybe a change in how 13 handles this?)

Now I'm trying to see if there's a way to tell the kernel to just ignore the UPS device so I don't have to get it to release first...

[LAN]

I know this thread is over a month old now, but I found a solution and wanted to report it. :)

I believe my problem stemmed from the fact that I'm using Multi-WAN, which then requires rules on the LAN side to set a gateway group for the outgoing traffic.

I have a rule above my GW Group rule that is just a generic "allow to firewall LAN IP"... but that overlooks multicast.

Had to add this rule, prior to my GW group selection rule:
source network: <LAN net>
destination host: 239.255.255.250  [multicast IP used for UPnP discovery]
Protocol: UDP
Port: 1900

This got UPnP functional again, at least with the handy Mac app "Port Map". I haven't tried an Xbox yet, but I suspect it will be OK too.

[did]

Hi all,

I have a LOT of experience with this.  Are both the OPNSense box & the other device on the same switch?  Are they on the same vlan?  Can they ping each other directly without going through a router?

Well, yes, sort-of. :)

all of my devices are on a single VLAN. I have a Ubiquiti EdgeSwitch 24-PoE as my "core", and in two locations I have Ubiquiti Unifi switches (an 8 port and a 16 port PoE). I have IGMP snooping disabled on the EdgeSwitch, and have not enabled it in the Unifi controller in the "Network" section, so everything should be getting multicast fllood. All systems that would normally be using UPnP are hardwired.

I did just realize that I had some level of IGMP filtering happening on my Unifi APs ("Multicast Enhancement" was enabled on the wireless network), which would partially explain why some of my testing was failing (laptop is on wifi)... but it doesn't explain why the Xbox wasn't working (which is hardwired).

However, there is one other piece in the game I often forget about... I have a "SamKnows" bandwidth testing box on my LAN at home. It's basically a small linux box that sits behind my opnsense box and it just bridges all traffic through it, and then runs internet speed tests when it detects enough of an "idle window" in the traffic flow. It's supposed to be a transparent bridge, although I've discovered that it filters 802.1q (i.e. it chokes on vlan tagging). Multicast had been working fine in the past though (UPnP has not been a problem before.)

I wouldn't be entirely surprised if something changed and it started filtering multicast though.  That said, after fixing the filtering on the wifi, I now can use the Windows UPnP test app that thecodemonk mentioned, and I'm getting responses from opnsense. So it doesn't seem like multicast is entirely blocked, at least. More testing needed to see if it will allow me to actually map a port though...

Win10 (in a VMware VM, using bridged networking on my Mac) with this UPnPTest app can map a port. 

The Mac itself cannot map a port or even get a response from opnsense via UPnP (although NAT-PMP works). I even pulled down the "upnpc" client  (which is part of the miniupnpd project, same thing used on opnsense) with homebrew, and it isn't even getting an answer from opnsense... which makes me wonder if Apple is doing something with multicast on recent versions of MacOS. I don't really care much about the Mac though, the xboxes are what need to work.

I need to give an xbox another try and see if it magically starts working now... I definitely have IGMP snooping disabled on the Unifi switch it is uplinked to, so it should be getting flooded, but it's probably worth verifying that with wireshark because Unifi gear is known to often do things you don't expect.

This is pfSense documentation, but AFAIK since OPNsense is still based on the same guts, the order of operations are the same...

https://docs.netgate.com/pfsense/en/latest/nat/process-order.html

I think you'd be able to feed the various loader.conf variables to the "loader" prompt, but you wouldn't be able to edit /etc/ttys from there (AFAIK).

So I think it would boot and give you all of the kernel boot output at that point, but the second it passed to userland and where you'd normally get a screen or login prompt, you'd have nothing (if /etc/ttys isn't setup correctly).

I could be wrong though...