OPNSense 16.1.1 + HardenedBSD 11-CURRENT

[lattera]

Hey All,

I've published a new build of OPNSense 16.1.1 with HardenedBSD 11-CURRENT! You can grab the build from here: https://hardenedbsd.org/~shawn/opnsense/hbsd-exp-08-16.1/

Future things to work on:

• Wireless isn't working. This is likely due to the new 802.11 stack in FreeBSD 11-CURRENT causing issues with the network interface code in OPNSense. Part of the problem is that the raw wireless device is now hidden from `ifconfig`.
• binary updates are now not supported. I've yet to get time to work on binary updates. So, to update to a future version, you'll need to do the usual config backup, reinstall, config restore.
• pfsync is still disabled. I'm unsure as to why this causes a kernel panic. If you are a C developer with time on your hands and want to tackle this, that'd be freaking awesome and very much appreciated.

For item #1, I've started work on getting wireless working with this commit: https://github.com/HardenedBSD/opnsense-core/commit/7c4dd2a6178343fa37880810ea94cadc141c0c78. I need to ping Adrian Chadd to figure out how to get the MAC address and the other bits from ifconfig that are now hidden that the network interface code expects without having to do a temporary clone of the device.

For item #2, OPNSense recently revamped how they provide binary updates for base. HardenedBSD now has an official binary updating mechanism as well (thanks G2, Inc for sponsoring the work!). Instead of using OPNSense's updating mechanism, I'd rather eat my own dogfood and use hbsd-update. More info about hbsd-update can be found here: https://hardenedbsd.org/article/shawn-webb/2015-12-31/introducing-hardenedbsds-new-binary-updater

For item #3, I'll need help with this one. If I were to tackle this, I'd first redo the build, but with FreeBSD 11-CURRENT instead of HardenedBSD 11-CURRENT and see if the behavior matches. If it does, then it's not a problem with HardenedBSD. If it doesn't, then the behavior is specific to HardenedBSD. I currently don't have the time it would take to do these steps, but I'd certainly love to help someone debug this if they have the time.

If anyone wants to dig into the pfsync issue, here's the crash I got:

Code Select Expand


Fatal trap 9: general protection fault while in kernel mode
cpuid = 3; apic id = 06
instruction pointer     = 0x20:0xffffffff82c22050
stack pointer           = 0x28:0xfffffe024b62aa60
frame pointer           = 0x28:0xfffffe024b62aaf0
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 12 (swi1: pfsync)


[weust]

Seems I'm having a busy night tomorrow.
Testing the 10.2 Hyper-V bug/patch, and this build. Though somehow I doubt that Hyper-V patch is in here, since it's only in 10.2-RELENG atm?

[franco]

If that one Hyper-V patch is the right one, it's on 11-CURRENT and 10-STABLE, but not on 10.2-RELEASE so far.

[weust]

Alright. Will find out tonight :-)

[interfaSys]

+

• bash is there
• git is there

-

• Wasn't able to use vidcontrol to set the resolution higher
• Many on screen backtraces. I know it doesn't help without actually showing the issues, but I just wanted to let others know. I think one of them was about if_rw if that exists? Lots of them about unionfs.
  

Apart from that, I ran the same netmap packet test I ran on 10.2 and there is the same realtek driver issue where the card locks up and stops processing packets.

[lattera]

-

• Wasn't able to use vidcontrol to set the resolution higher
• Many on screen backtraces. I know it doesn't help without actually showing the issues, but I just wanted to let others know. I think one of them was about if_rw if that exists? Lots of them about unionfs.
  

Apart from that, I ran the same netmap packet test I ran on 10.2 and there is the same realtek driver issue where the card locks up and stops processing packets.

I'm not sure why vidcontrol wouldn't work. Since I only use OPNSense on physical appliances that only have a serial port, it's not something I've tested. Keep in mind that these builds are mostly for my own use. I publish them in the hopes that someone else might find them useful. ;-)

The backtraces you're getting are safe to ignore. They're there because the WITNESS option is enabled, causing the kernel to check for something called a "lock order reversal." They're more for developers eyes only. WITNESS is a good option to have enabled for security, but it does come at the cost of performance.

As far as your netmap issues are concerned, I have zero experience and knowledge in that area. If you experience it on official OPNSense builds, that means it's not an issue with HardenedBSD. But it's also good to know that it happens on both 11-CURRENT and 10.2-RELEASE.

[interfaSys]

I'm not sure why vidcontrol wouldn't work. Since I only use OPNSense on physical appliances that only have a serial port, it's not something I've tested.

I'm wondering if that has something to do with the change from syscons to vt4. But most probably something for the OPNsense team to handle

Keep in mind that these builds are mostly for my own use. I publish them in the hopes that someone else might find them useful. ;-)

I think it's great. Gives me an opportunity to test 11 from time to time until things have stabilised. After a few runs, I'll probably manage to have a working config and will be able to keep it running for longer. That makes it easier to test stuff.

The backtraces you're getting are safe to ignore. They're there because the WITNESS option is enabled, causing the kernel to check for something called a "lock order reversal." They're more for developers eyes only. WITNESS is a good option to have enabled for security, but it does come at the cost of performance.

Got it :)

As far as your netmap issues are concerned, I have zero experience and knowledge in that area. If you experience it on official OPNSense builds, that means it's not an issue with HardenedBSD. But it's also good to know that it happens on both 11-CURRENT and 10.2-RELEASE.

Yep, I wouldn't worry about that. It's already been reported to Netmap, FreeBSD and Realtek. We'll see if something changes, but this probably has a very low priority. I had tested the drivers from 11 before, but wanted to see if the kernel made a difference. The only difference I saw was a loss of performance, probably due to WITNESS then, but in the end the same lock up happened.

Thanks for your work :)