OPNsense Forum

Archive => 18.1 Legacy Series => Topic started by: franco on October 27, 2017, 05:30:21 PM

Title: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on October 27, 2017, 05:30:21 PM
Hi guys,

As a twist to our normal beta images, this time around we simply offer a manual upgrade of the operating system code, which means your GUI release version stays and follows 17.7.x if you want to.

All the updates and commands found here are signed and verified to allow a secure and stable update experience as usual. :)

On top of our HardenedBSD additions this new OS version for OPNsense 18.1 received the following improvements:

o Shared forwarding now works on IPv6, in conjunction with "tryforwarding" and better reply-to multi-WAN behaviour
o A potential fix for the high CPU load of the SNMP hostres module
o Realtek vendor NIC driver version 1.94
o FreeBSD 11.1, see https://www.freebsd.org/releases/11.1R/relnotes.html

The move to this new OS version is simple:

# opnsense-update -bkgr 18.1.b -n "snapshots\/beta"
# opnsense-update -L
# /usr/local/etc/rc.reboot

To explain, we are installing the b(ase), (debu)g k(ernel), r(elease) 18.1-BETA from mirror (locatio)n snapshots/beta.

The second command locks the base and kernel to prevent upgrades to switch back to 17.7.1.

The third command, as you would have guessed, issues the reboot.

From an operational standpoint, we've seen no differences to FreeBSD 11.0 and the behaviour is the same, but would like you to help pinpoint potential issues and hiccups as covering all use cases and hardware can only be a team effort.

Should there be any need to switch back to 11.0 / 17.7 you can run the following to revert your system:

# opnsense-update -U
# opnsense-update -bkd
# /usr/local/etc/rc.reboot

Unlock the kernel and base, update standard k(ernel) and b(ase) from the d(efault) location, and reboot.

That's all, have fun!


Thanks,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: DanMc85 on October 28, 2017, 02:49:36 AM

opnsense-update -bkgr 18.1.b -n "snapshots\/beta"

This command isn't working properly for me.  It Fetches the first file ****.obsolete... done  the second file *****.txz comes up with an opnsense-verify error 04091068 rsa routines  bad signature.  Signature is not valid.

Any ideas?

My current build:
OPNsense 18.1.a_291-amd64
FreeBSD 11.0-RELEASE-p12
OpenSSL 1.0.2l 25 May 2017
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on October 28, 2017, 03:41:36 PM
Which mirror are you using?

Since the first one goes through this could mean thee things in order of increasing possibility:

1. The mirror has a faulty set file.
2. Your download was truncated by your network.
3. There was not enough space on your OPNsense to store the full set.


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: tillsense on October 28, 2017, 06:59:25 PM
Hi Franco,

a proxmox vm has been stable for 10h.
...
Kernel locked at 18.1.b-amd64, skipping.
Base locked at 18.1.b-amd64, skipping.


OPNsense 18.1.a_292-amd64
FreeBSD 11.1-RELEASE-p2
OpenSSL 1.0.2l 25 May 2017

cheers till
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Solaris17 on October 28, 2017, 07:41:17 PM
Followed the directions and still ended with

OPNsense 17.7.7_1-amd64
FreeBSD 11.1-RELEASE-p2
OpenSSL 1.0.2l 25 May 2017


That doesnt seem right? Attempting to do it again after reboot shows that I am "upto date skipping"

Though the OP does say

QuoteAs a twist to our normal beta images, this time around we simply offer a manual upgrade of the operating system code, which means your GUI release version stays and follows 17.7.x if you want to.

others report GUI change of 18.x ?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: tillsense on October 28, 2017, 08:10:07 PM
@Solaris17 @all

17.7.x is ok i'm in addition on the opnsene devel package over git.

cheers till
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Ren on October 29, 2017, 06:07:39 AM
I just updated to 18.1 and Opnsense Version number did not change. Is it suppose to change to 18.1 ? FreeBSD Version got updated to 11.1 and i did get driver version 1.94.01 for my realtek NICS


OPNsense 17.7.7_1-amd64
FreeBSD 11.1-RELEASE-p2
OpenSSL 1.0.2l 25 May 2017
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: phoenix on October 29, 2017, 08:13:11 AM
Quote from: Ren on October 29, 2017, 06:07:39 AM
I just updated to 18.1 and Opnsense Version number did not change. Is it suppose to change to 18.1 ?
No, it's not supposed to change and the answer is in the first paragraph of the first post in this thread. :)
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on October 29, 2017, 10:28:26 AM
Hi,

Sorry for not being more clear... OPNsense version stays at 17.7.x, if you lock the kernel and base you'll simply stay on 11.1 instead of going back to 11.0 on updates.

18.1.x is reached via:

# opnsense-update -t opnsense-devel

But at this point there is not a lot of interesting things in there as most of it went to 17.7.x already.

To switch back, type:

# opnsense-update -t opnsense


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Ren on October 29, 2017, 04:47:03 PM
Quote from: phoenix on October 29, 2017, 08:13:11 AM
Quote from: Ren on October 29, 2017, 06:07:39 AM
I just updated to 18.1 and Opnsense Version number did not change. Is it suppose to change to 18.1 ?
No, it's not supposed to change and the answer is in the first paragraph of the first post in this thread. :)

Oh ok. I think i glossed over it last night. Thank you
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Ren on October 29, 2017, 04:48:21 PM
Quote from: franco on October 29, 2017, 10:28:26 AM
Hi,

Sorry for not being more clear... OPNsense version stays at 17.7.x, if you lock the kernel and base you'll simply stay on 11.1 instead of going back to 11.0 on updates.

18.1.x is reached via:

# opnsense-update -t opnsense-devel

But at this point there is not a lot of interesting things in there as most of it went to 17.7.x already.

To switch back, type:

# opnsense-update -t opnsense


Cheers,
Franco

Thanks for the info
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on October 29, 2017, 10:40:04 PM
I have installed this new Kernel on a real hardware.
Where are we supposed to pay attention ?
Thank you
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on October 30, 2017, 09:13:25 AM
Hi Julien,

Apart from the particular points stated above there is no new test-worthy addition. Just look out for stability issues. But as I said, it's been tested for a while now and has shown good quality.


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on October 30, 2017, 09:31:22 PM
Quote from: franco on October 30, 2017, 09:13:25 AM
Hi Julien,

Apart from the particular points stated above there is no new test-worthy addition. Just look out for stability issues. But as I said, it's been tested for a while now and has shown good quality.


Cheers,
Franco
I am working on it and I agree performance is great,
I will wait a week and go for it on productions with one site and check the behaivor.
today i've recieved two updates which i installed them however after i check for the updates they pop ups again.
i have rebooted the appliance twice but it still shows up.
thought i'd share here
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on October 31, 2017, 09:05:37 AM
Yes, you are running opnsense-devel and I added kernel/base support there. But you can't install the 17.7.1 kernel and base because you locked them to stay at 18.1-BETA with "opnsense-update -L" so everything is as it should be. :)


Thanks,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on October 31, 2017, 09:15:05 PM
Quote from: franco on October 31, 2017, 09:05:37 AM
Yes, you are running opnsense-devel and I added kernel/base support there. But you can't install the 17.7.1 kernel and base because you locked them to stay at 18.1-BETA with "opnsense-update -L" so everything is as it should be. :)


Thanks,
Franco
Thank you for the explanation,
let test .
i'll report back in case something noticed.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: lattera on November 01, 2017, 06:29:00 PM
I migrated my OPNsense firewall at work from 17.7 to 18.1. Working great so far. Suricata is in IPS mode.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: monstermania on November 02, 2017, 05:11:40 PM
I've installed 17.7.5 and update to 18.1 and FreeBSD 11.1 on a Securepoint RC100 (Lexcom 3I525D) with Realtek NIC's.
So far without any problems.  :)



Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: lattera on November 02, 2017, 05:48:27 PM
I just got around to testing my Tor-ified setup and all is well there, too.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on November 03, 2017, 05:38:56 PM
Hi Lattera,
are you using VLANS on your productions ?
Are you using Promiscuous mode ?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: lattera on November 03, 2017, 05:47:06 PM
Quote from: Julien on November 03, 2017, 05:38:56 PM
Hi Lattera,
are you using VLANS on your productions ?
I was curious why IPS mode ?

I don't use VLANs currently. I use Suricata in IPS mode to help increase security.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on November 08, 2017, 05:59:28 PM
Hi Franco,
I have installed this on a new Hardware.
root@firewall:~ # opnsense-update -bkgr 18.1.b -n "snapshots\/beta"
Kernel locked at 18.1.b-amd64, skipping.
Base locked at 18.1.b-amd64, skipping.
Your system is up to date.

but on the Gui it shows the version
OPNsense 17.7.7_1-amd64
FreeBSD 11.1-RELEASE-p2
OpenSSL 1.0.2l 25 May 2017

Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on November 08, 2017, 06:08:23 PM
Did you change the firmware GUI settings? I used a stale mirror link snapshots/beta to prevent this foot-shooting. ;)

Change it back to normal. And btw there is no update so far from 17.7.7.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on November 08, 2017, 09:38:51 PM
Quote from: franco on November 08, 2017, 06:08:23 PM
Did you change the firmware GUI settings? I used a stale mirror link snapshots/beta to prevent this foot-shooting. ;)

Change it back to normal. And btw there is no update so far from 17.7.7.
Hi Franco,
I understand there is no update 17.7.7 and the 18.1 is a beta.
we have followed the same steps on hardware 1 and its updated and shows 18.1
Firmware Mirror and Flavour is still Default.
I just want to share this with you as I don't consider this a problem  :) just sharing the outcome.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on November 09, 2017, 01:19:06 AM
Now it looks ok, but you edited away the previous error which makes this discussion hard to follow from now on. ;)
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: va176thunderbolt on November 15, 2017, 02:56:53 PM
I've updated my home machine (I work from home, so it gets a good workout).

And A8-5545M
8GB RAM
3 Realtec rtl8111 nics

The guy is noticeably faster now.

I have Suricata 4.0.1 running, and I have it watching native and clan interfaces. No issues, has been very stable - very good job!
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on November 15, 2017, 03:15:22 PM
Yay, thanks. I agree that FreeBSD did a good job on this release. The final 18.1 system will not be very different from the beta from the looks of it. :)


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: interfaSys on November 17, 2017, 11:19:42 PM
Unusable when Suricata in IPS mode (+ promiscuous) is enabled on VLANs. This is on a Zotac CI323 with Realtek chips.
Endless reboots until Suricata is turned off.

Couldn't find anything in dmesg, so it seems to be a different issue than the kernel crashes that used to happen.

On a more positive note, FreeBSD 11.1 seems to boot normally on that hardware. It used to be that the card reader would hang the boot process for 1-2 minutes.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Ren on November 20, 2017, 08:31:24 PM
Quote from: interfaSys on November 17, 2017, 11:19:42 PM
Unusable when Suricata in IPS mode (+ promiscuous) is enabled on VLANs. This is on a Zotac CI323 with Realtek chips.
Endless reboots until Suricata is turned off.

Couldn't find anything in dmesg, so it seems to be a different issue than the kernel crashes that used to happen.

On a more positive note, FreeBSD 11.1 seems to boot normally on that hardware. It used to be that the card reader would hang the boot process for 1-2 minutes.


I'm running the same system with 4GB of ram and did not experience any reboots. However my WAN connection speed dropped to 20Mbps from 70Mbps. If i remove my VLAN, and OPENVPN interfaces from the HOME NETWORK tab (only LAN defined) in SURICATA my connection speeds jumps up to 50Mbps. What is the expected performance hit when running Suricata ?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on November 21, 2017, 12:19:06 PM
Quote from: Ren on November 20, 2017, 08:31:24 PM
Quote from: interfaSys on November 17, 2017, 11:19:42 PM
Unusable when Suricata in IPS mode (+ promiscuous) is enabled on VLANs. This is on a Zotac CI323 with Realtek chips.
Endless reboots until Suricata is turned off.

Couldn't find anything in dmesg, so it seems to be a different issue than the kernel crashes that used to happen.

On a more positive note, FreeBSD 11.1 seems to boot normally on that hardware. It used to be that the card reader would hang the boot process for 1-2 minutes.


I'm running the same system with 4GB of ram and did not experience any reboots. However my WAN connection speed dropped to 20Mbps from 70Mbps. If i remove my VLAN, and OPENVPN interfaces from the HOME NETWORK tab (only LAN defined) in SURICATA my connection speeds jumps up to 50Mbps. What is the expected performance hit when running Suricata ?
I am experiencing the same, the internet drop really from 1Gbps to 300/400 Mbps with my Intel(R) PRO/1000 Network Connection 7.6.1-k
I have disabled the IPS for now
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: interfaSys on November 21, 2017, 10:01:14 PM
Found this in the logs today:

Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 06
fault virtual address = 0xc
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff8070b1b4
stack pointer         = 0x28:0xfffffe0232213280
frame pointer         = 0x28:0xfffffe02322132a0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 31124 (W#01-re0)
version.txt06000016713203656554  7630 ustarrootwheelFreeBSD 11.1-RELEASE-p2 #0 c967ed374(master): Tue Oct 17 20:39:21 CEST 2017
    root@sensey64:/usr/obj/usr/src/sys/SMP


Must be from when Suricata was enabled.

Full crash dump is herE:
https://paste.ubuntu.com/26014885/
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Lumpy on November 23, 2017, 10:55:56 AM
Today it's showing 17.7.8 is newer than 18.1_b? Anything I've missed?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: phoenix on November 23, 2017, 11:12:07 AM
Quote from: Lumpy on November 23, 2017, 10:55:56 AM
Today it's showing 17.7.8 is newer than 18.1_b? Anything I've missed?
No, I'm guessing it's because you've updated all the packages apart from the kernel. I've just done the same upgrade and, obviously, it's showing the same thing. :)
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on November 23, 2017, 11:44:07 AM
On opnsense-devel, there is an improvement that will show out of sync kernel and base sets. Any mismatch is shown there, 17.7.1 before, now 17.7.8, because that is our current release. That was the reason for "opnsense-update -L" and preventing the firmware to go back to its know good version. :)

You can either "opnsense-update -U" and go back to the latest FreeBSD 11.0 system or stay on 11.1, depending on your wariness for the two kernel issues that FreeBSD published (see our release notes).


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Lumpy on November 24, 2017, 10:18:31 AM
I think I've got a problem with understanding what excatly you mean. I used

# opnsense-update -bkgr 18.1.b -n "snapshots\/beta"
# opnsense-update -L
# opnsense-update -t opnsense-devel
# /usr/local/etc/rc.reboot

I'm on "OPNsense 18.1.a_364-amd64" now, if I check for updates via the webinterafce I get two updates (see attachment). Is this the expected behaviour?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: phoenix on November 24, 2017, 10:53:27 AM
What does it do/show if you run the following command:

opnsense-update -bkgr 18.1.b -n "snapshots\/beta"
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Lumpy on November 24, 2017, 11:20:57 AM
Kernel locked at 18.1.b-amd64, skipping.
Base locked at 18.1.b-amd64, skipping.
Your system is up to date.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: phoenix on November 24, 2017, 11:24:10 AM
There you go, that confirms that you're on the version you expect to be on. ;)
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Lumpy on November 24, 2017, 11:32:02 AM
Thank you :) Got a bit irritated though because I thought I've blocked the downgrade correctly. I didn't know that it won't work for the webinterafce.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: interfaSys on November 24, 2017, 01:37:56 PM
Quote from: Ren on November 20, 2017, 08:31:24 PM
I'm running the same system with 4GB of ram and did not experience any reboots.

Which BIOS are you running?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Ren on November 25, 2017, 07:41:43 PM
Quote from: interfaSys on November 24, 2017, 01:37:56 PM
Quote from: Ren on November 20, 2017, 08:31:24 PM
I'm running the same system with 4GB of ram and did not experience any reboots.

Which BIOS are you running?

The latest BIOS which is Version 2K170307
Link https://www.zotac.com/us/files/download/by_product?p_nid=501278&driver_type=238&os=All
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Julien on November 27, 2017, 03:43:22 PM
Quote from: Lumpy on November 24, 2017, 10:18:31 AM
I think I've got a problem with understanding what excatly you mean. I used

# opnsense-update -bkgr 18.1.b -n "snapshots\/beta"
# opnsense-update -L
# opnsense-update -t opnsense-devel
# /usr/local/etc/rc.reboot

I'm on "OPNsense 18.1.a_364-amd64" now, if I check for updates via the webinterafce I get two updates (see attachment). Is this the expected behaviour?

Yes, you are running opnsense-devel and I added kernel/base support there. But you can't install the 17.7.8 kernel and base because you locked them to stay at 18.1-BETA with "opnsense-update -L" so everything is as it should be.
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: Lumpy on November 28, 2017, 08:08:59 AM
And what should be done to upgrade to the neweset revision auf 18.1b? Which command should I use?
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: franco on November 30, 2017, 03:22:04 PM
Easy, there is no "neweset revision of 18.1.b". The next release will be 18.1-RC1. :)


Cheers,
Franco
Title: Re: [CALL FOR TESTING] FreeBSD 11.1, newer Realtek vendor driver and more
Post by: guywyers on November 30, 2017, 07:01:53 PM
Hey guys,

Just started configuring OPNsense on a new hardware config and run into the problem documented here:  https://forums.freebsd.org/threads/59627 (https://forums.freebsd.org/threads/59627)
It's not life threatening but annoying, because it means that every time you do a shutdown, the system hangs in the last phase of the power-off and requires a physical power-off/power-on cycle to get back in the normal state.
This is definitely a pain if you do reboots as a consequence of configuration changes.
As explained in the link above, the FreeBSD workaround is to use  shutdown -r now and they also mention that the problem is fixed in FreeBSD 11.1.

So, seeing that this beta contains FreeBSD 11.1, I installed it and indeed the problem is fixed. Just wanted to bring it up in case other people run in to this.

Cheers.