Topics

1

22.7 Legacy Series / 3rd party tool turbostat crashes with "Floating exception (core dumped)"

« on: September 01, 2022, 07:53:32 pm »

For testing the current CPU clock, I have been running turbostat for many years. This has worked over many versions of FreeBSD. Since version 13 the tool stops after startup with "Floating exception (core dumped)".

Here's what I do to install and run turbostat (worked so well before FreeBSD):

Code: [Select]

pkg add https://pkg.freebsd.org/FreeBSD:13:amd64/latest/All/turbostat-4.17_2.pkg
rehash
kldload cpuctl
turbostat --interval 3


Since FreeBSD 13 I now have the following output:

Code: [Select]

# turbostat --interval 3
turbostat version 17.06.23 - Len Brown <lenb@kernel.org>
CPUID(0): AuthenticAMD 13 CPUID levels; family:model:stepping 0xf:30:1 (15:48:1)
CPUID(1): SSE3 MONITOR - - - TSC MSR - -
CPUID(6): APERF, No-TURBO, No-DTS, No-PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, No-EPB
CPUID(7): No-SGX
NSFOD /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
Floating exception (core dumped)

My questions:

• Is there any reason that turbostat should no longer run with FreeBSD 13? (When installing I get a message that the package is no longer maintained.)
• Are there alternative similar tools that show the current CPU clock?

Thanks for any hints.

2

Virtual private networks / Temporarily accept an expired server certificate in the OpenVPN client?

« on: March 04, 2021, 09:38:50 pm »

In an OpenVPN Road warrior scenario, when the server TLS certificate expires, is it possible to (temporarily) disable the expiration date checking or the whole server certificate checking on the client side?

3

Hardware and Performance / [SOLVED] APU - Is your CPU clock stuck at low frequency after some time?

« on: January 30, 2021, 12:09:42 pm »

The APU2/3/4 devices are predestined for use as routers under OPNsense: Inexpensive reliable hardware with open source BIOS. Accordingly, it is often used by users here in the forum.

The performance of the used SOC AMD GX412TC is certainly much lower compared to i3 or i5 systems. However, I can say from experience that they are more than sufficient in many cases and more performance is simply not necessary. And that's why I prefer such a cost-effective and power-saving device whenever possible.

However, questions have been piling up here and in other forums over the past few months as to why APU-based firewall performance dips and doesn't seem to be enough for some, while it's fast enough for others in a similar context.

Of course, this always depends on the individual case: What services are enabled on the firewall? Does WAN have to be connected via PPPoE? How large is the MTU? Are the correct tunables set? Etc.

Meanwhile I am not sure if there is not one more aspect: I have two APU2 in use (APU2C4 and APU2E4). I noticed that they are really responsive and powerful after a reboot. After some time (several hours), they become noticeably sluggish and the CPU utilization suddenly seems higher.

I then noticed that after a reboot, the APU can scale up the frequency to the nominal maximum, which is 1GHZ. After a while, however, the maximum achievable frequency seems to be reduced to 600MHz. This then remains the same until a reboot. After the reboot, the 1GHz can be reached again for a certain time. Until suddenly the 600MHz limit applies again.

I would be interested to know how the situation looks like with your APU. If you want to participate, you can post your observations here. In the following I describe how you can determine the maximum busy clock.


You need console or SSH access to your OPNsense:

All the measurements are done on the console or in an SSH shell on the OPNsense. If you do not have access to the console, you can set up SSH access as follows:

• System: Settings: Administration
• Secure Shell Server: Enable Secure Shell
• Root Login: Permit root user login
• Authentication Method: Permit password login
• Now you should be able to access with ssh YOUR_USER@YOUR_FIREWALL_IP
• If you do not access with root, you may have to become super user: su

Note: After taking the measurements, the access can be deactivated again for security reasons.


You need to install and use the tool turbostat:

The measurements are done with the tool turbostat, which can be installed as follows:

Code: [Select]

pkg add http://pkg0.isc.freebsd.org/FreeBSD:12:amd64/latest/All/turbostat-4.17_2.txz
rehash
Before using turbostat you have to load the kernel module cpuctl once before doing measurements:

Code: [Select]

kldload cpuctl

A measurement series is started as follows:

Code: [Select]

turbostat --interval 3
After that the tool prints the CPU statistics every 3 seconds.

After a reboot everything runs normally and the output shows that the Bzy_MHz is near 1GHz:

Code: [Select]

root@router:~ # turbostat --interval 3
turbostat version 17.06.23 - Len Brown <lenb@kernel.org>
turbostat: /dev/cpuctl0 missing, kldload cpuctl: No such file or directory
root@router:~ # kldload cpuctl
root@router:~ # turbostat --interval 3
turbostat version 17.06.23 - Len Brown <lenb@kernel.org>
CPUID(0): AuthenticAMD 13 CPUID levels; family:model:stepping 0xf:30:1 (15:48:1)
CPUID(1): SSE3 MONITOR - - - TSC MSR - -
CPUID(6): APERF, No-TURBO, No-DTS, No-PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, No-EPB
CPUID(7): No-SGX
NSFOD /sys/devices/system/cpu/cpu1/cpufreq/scaling_driver
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       355     38.76   915     998     170
0       0       247     28.12   880     998     32
1       1       222     25.44   871     998     69
2       2       414     44.66   928     998     43
3       3       536     56.82   943     998     26
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       313     34.39   910     998     334
0       0       410     44.19   928     998     184
1       1       336     36.68   915     998     80
2       2       236     26.66   885     998     49
3       3       269     30.02   898     998     21
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       267     29.60   904     998     247
0       0       520     54.88   947     998     68
1       1       289     31.59   914     998     82
2       2       127     15.56   813     998     42
3       3       135     16.36   825     998     55
^C

When the OPNsense has been running for a few hours, the output shows that the Bzy_MHz is below 600MHz (even under maximum load):

Code: [Select]

root@router:~ # turbostat --interval 3
turbostat version 17.06.23 - Len Brown <lenb@kernel.org>
CPUID(0): AuthenticAMD 13 CPUID levels; family:model:stepping 0xf:30:1 (15:48:1)
CPUID(1): SSE3 MONITOR - - - TSC MSR - -
CPUID(6): APERF, No-TURBO, No-DTS, No-PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, No-EPB
CPUID(7): No-SGX
NSFOD /sys/devices/system/cpu/cpu1/cpufreq/scaling_driver
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       52      8.61    599     998     162
0       0       78      13.01   599     998     37
1       1       37      6.21    599     998     69
2       2       44      7.40    599     998     35
3       3       47      7.82    599     998     21
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       57      9.56    598     997     208
0       0       87      14.48   598     996     51
1       1       45      7.53    598     996     79
2       2       55      9.15    599     998     62
3       3       42      7.09    599     998     16
Core    CPU     Avg_MHz Busy%   Bzy_MHz TSC_MHz IRQ
-       -       59      9.81    599     999     297
0       0       94      15.67   600     1000    67
1       1       45      7.56    600     1000    104
2       2       60      9.97    599     998     68
3       3       36      6.01    599     998     58
^C

When you report your observations, it would be interesting to know which BIOS version you have installed (can be conveniently viewed in the Hardware Information widget) and whether you have the Core Performance Boost feature set to enabled or disabled in the BIOS (in newer BIOSes, the default value is enabled).

4

Tutorials and FAQs / HOWTO operate the console of the APU via the network (using ser2net)

« on: January 29, 2021, 09:42:46 pm »

The PC Engines APU devices are a cost-effective, stable and reliable hardware for a firewall based on OPNsense. Accordingly, I have two of them in use.

The APUs are so-called headless systems. I.e. they have no monitor output but are initially connected via the serial port. This works fine and once OPNsense is up and running, console access is usually no longer necessary, since OPNsense can be managed completely via the web UI (and occasionally ssh for very special needs).

In very rare cases, however, it is necessary to access the console, e.g. to change the BIOS settings or if a manual intervention is necessary after an unsuccessful update (but I have never had to do this).

Of course, in such a rare case you can go with the notebook close to the firewall and temporarily access the console with a Serial2USB cable. I have done this in the past maybe once a year at most and otherwise tried to prevent it, also because it was a bit tedious.

Recently I saw on the web that you can easily create a telnet access to a serial port with the tool ser2net. Usually you can find instructions for a Raspberry Pi. But this works for all such single board computers. I had an old Orange Pi PC lying around, which I converted to my Serial2Network device:

• install armbian
• install ser2net as an autostarting service
• configure ser2net
• access it with telnet [IP] [portnumber

The Orange Pi PC is now sitting in our tech basement on top of the APU. It takes power from the USB port on the APU. And the Serial2USB cable from PC Engines connects the console of the APU to the USB port of the Orange Pi PC.

Here I came across the possibility and it is described: https://www.jpaul.me/2019/01/how-to-build-a-raspberry-pi-serial-console-server-with-ser2net/

5

20.7 Legacy Series / Firewall Logs Widget - repeats the same log entries multiple times

« on: January 26, 2021, 07:59:46 pm »

OPNsense 20.7.8-amd64

Today I noticed, that the "Firewall Logs" Widget showed many entries. But the firewall log itself (e.g. plain view) only contains two entries.

When opening the dashboard page, initially there are only two entries. But every second these two entries are added again and again until the maximum of entries in the widget is reached.

Is this a known problem?

6

General Discussion / [SOLVED] Plugin os-ntopng - starts but Web-GUI not accessible

« on: January 17, 2021, 11:10:39 am »

Hello everybody

I tried installing the os-ntopng plugin from OPNsense yesterday. So not the newly offered one from ntopng itself.

Redis had been installed for a while on my current OPNsense system (OPNsense 20.7.7_1-amd64) and was running fine. The installation of os-ntopng went normally. The service became green.

Before accessing the GUI of ntopng I first looked at the settings of ntopng in the OPNsense GUI and saw under Advanced that you can remove the default value of LAN if you want to configure the interfaces later in ntopng. Then I pressed Save button and restarted the ntopng service.

When I finally wanted to access ntopng GUI (LAN IP of the OPNsense + port number from the settings), the browser turned only to the timeout. The same is true for http and https ports.

I can't exclude that my (premature) removal of the LAN interface on level OPNsense plugin settings led to this problem.

Now I have already uninstalled both redis and ntopng plugins several times, additionally manually deleted the residues (conf, db) from the system and reinstalled the plugins. Unfortunately always with the same success: ntopng seems to run. But I can't reach its GUI.

In order to make a completely fresh installation, do I need to uninstall certain files?

Thanks for any hints. :-)

Output of ntopng restart:

Code: [Select]

# /usr/local/etc/rc.d/ntopng restart
Stopping ntopng.
Waiting for PIDS: 90421.
Starting ntopng.
17/Jan/2021 11:00:52 [Ntop.cpp:2291] Setting local networks to 127.0.0.0/8
17/Jan/2021 11:00:52 [Redis.cpp:162] Successfully connected to redis 127.0.0.1@0
17/Jan/2021 11:00:52 [Redis.cpp:162] Successfully connected to redis 127.0.0.1@0
17/Jan/2021 11:00:53 [Ntop.cpp:2231] Parent process is exiting (this is normal)


7

Hardware and Performance / With performant hardware - bare metal or Proxmox as base?

« on: January 16, 2021, 12:57:44 pm »

Hi everybody

I have been running OPNsense successfully on an APU2c4 for quite some time. Absolutely stable system.

Now I want to make my firewall fit, if I should have GBit internet one day and so I can run some more services performant:
- Blocklists
- Suricata IDS
- maybe Sensei (only analysis and reporting)

On the shortlist is a Yanling Hardware appliance (from Aliexpress) with 6 Intel NIC, Intel i5-8250U CPU, 16GB RAM, 500GB SSD. (Almost) identical in construction to Protectli FW6D. Now I wonder if I should install OPNsense directly on the hardware, or if I should install Proxmox as a base, so that I can benefit even better from the great performance of the hardware.

Advantages would be:
- Console reachable over the LAN
- Snapshot possibility before updates or an experimental change
- Quick jump of a backup VM in case of problems

Looking at the forum, I see a mixed picture: Some swear by Proxmox as a base and have a stable system. Others struggle with stability and/or performance.

Those who have gone the Proxmox route:
- What are your experiences?
- If Proxmox: Which VM setup would you recommend? (Which CPU, which NIC, RAM settings etc.)

What do I need to consider when setting up Proxmox so that the WAN port is not vulnerable via Proxmox? Should I best choose pass-through of the WAN NIC to the OPNsense VM instead of virtual bridge?

Thanks a lot for your inputs!
Tom

8

20.1 Legacy Series / Dual WAN - WAN2 should really be used only in case WAN1 is dead (WAN2 metered)

« on: April 18, 2020, 10:05:24 pm »

Hello everybody

I am a feeder for LiveATC.net and would like to achieve high availability of my stream. Therefore I have attached a cheap TPLink 4G router (WAN2) to the OPNsense next to my main cable modem (WAN1) with a flat rate. The SIM for WAN2 is coupled with a prepaid subscription, where the data volume does not expire but costs per MB used.

Now I have configured OPNsense for dual WAN as described in the Doc (https://docs.opnsense.org/manual/how-tos/multiwan.html). That means especially with the fixed assignment of one DNS server each for WAN and for WAN2.

The setup works. But due to the monitoring ping and possibly DNS resolutions on WAN2, there is permanently a minimal load. Not much - but enough so that the prepaid subscription is used down to zero every three months.

Is there any way I can ensure that WAN2 is not at all used during normal operation (when WAN1 normally available)? Only in case of WAN1 failure, the traffic should go over WAN2 without WAN2 being permanently used for pings and DNS queries.

Thanks & best regards
Thomas

9

Hardware and Performance / Optimized performance of my PC Engines APU2c4

« on: August 20, 2019, 09:52:45 pm »

My cable provider offers 100mbit/s down and 10mbit/s up.

With my current OPNsense configuration running on an PC Engines APU2c4 I couldn't use the full speed of my provider (75mbit/s down and 10mbit/s up only).

During a research I came across the following instructions for pfsense:
https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/

I followed the instructions for my OPNsense and made the changes. After a reboot I was astonished: Now my firewall reaches the maximum throughput of my provider.

So I recommend all users of an APU to optimize the settings.

10

19.7 Legacy Series / After Update to 19.7.2 Service "flowd_aggregate Insight Aggregator" is stopped

« on: August 05, 2019, 07:15:34 pm »

Hello

Today i updated from 19.7.1 to 19.7.2 on my APU2c4. Update itself run without problems.

But after the update the Service "flowd_aggregate" was stopped and could not be started again.

In the menu "Reporting: NetFlow" I could not reapply the current settings there, as an error stated that the WAN interface was missing in Listening Interfaces (it really was missing).

After manually re-adding the WAN interface there, I could apply the settings. In the dashboard the Service "flowd_aggregate" was shown green/running. But after a refresh of the dashboard, the service is showed as stopped again.

In the general log I found:

/flowd_aggregate.py: flowd aggregate died with message Traceback (most recent call last): File /usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 160, in run aggregate_flowd(self.config, do_vacuum) File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 80, in aggregate_flowd stream_agg_object.add(copy.copy(flow_record)) File "/usr/local/opnsense/scripts/netflow/lib/aggregates/source.py", line 117, in add super(FlowSourceAddrDetails, self).add(flow) File "/usr/local/opnsense/scripts/netflow/lib/aggregates/__init__.py", line 185, in add self._update_cur.execute(self._update_stmt, flow) sqlite3.DatabaseError: database disk image is malformed

What caused this issue?
What is the best thing to resolve this issue?

Thanks!

Tom

11

18.7 Legacy Series / Traffic Graph: Wrong scale on Out?

« on: September 18, 2018, 09:33:23 pm »

Hi everybody,

I observe strange traffic graphs that seem not to be consistent between interfaces and that do not match the transferred total data volume. To me it seems (at least sometimes) as the scale of the "Out" Graph is to high by factors.

Attached are images showing the graphs of
- the Traffic Graph Widget
- the Traffic Graph in Reporting

During the shown period there was only marginal traffic. But the LAN Out curve looks like that there has been sent data from OPNsense to the LAN. But the small detail graphs on the Reporting - Traffic Graph page show, that there is no such data stream. Even when comparing the high LAN Out graph to the low total amount of data transferred during the period, it becomes obvious that the total graph can not show the truth.

OPNsense version: I saw this behavior with 18.7.2 and 18.7.3. Maybe the problem was there before.
Hardware: APU2c4

Any thoughts or ideas?

Thanks,
Thomas

12

Development and Code Review / About the testing of OPNsense

« on: September 23, 2017, 04:53:16 pm »

Hello everybody,

first of all: many thanks to all of you contributing to OPNsense. I am impressed how clean and logical everything is presented. Really a great UI and functionality!

I am currently evaluating a firewall for my home installation. The most important things for a firewall to me are: 1. security and 2. reliabilty.

I am software engineer and like build and test automation a lot. One thing I asked myself is: how are releases of OPNsense built and tested? I could not find anything about the testing process. Is there some kind of an automated build and testing process? Maybe someone can give some insights.

Thanks & kind regards,
Tom

13

17.7 Legacy Series / APU2C4 - Usage of the onboard LEDs

« on: September 06, 2017, 09:32:45 pm »

Hello everybody

I have assembled an APU2C4 from PC Engines in a nice red case and have successfully installed OPNsense 17.7.1.

Everything went smooth. System is up an running just perfect. An ideal combination - quality hardware with quality software. Thanks to everybody making this possible.

The APU has 3 LEDs on its front. One is showing power status. But the two other LEDs remain dark/unuesed. These could be used e.g. for warnings, gateway status etc. I could not find any information about the usage of the LEDs.

Is there a way to use the LEDs in OPNsense?

Thanks
Thomas