Call for testing: netmap on 20.7

[actionhenkt]

Im not able to update repoL

root@SRV190520:~ # pkg install os-sensei
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
pkg: https://updates.sunnyvalley.io/opnsense/FreeBSD:12:amd64/OpenSSL/repo/meta.txz: Not Found
repository SunnyValley has no meta file, using default settings
pkg: https://updates.sunnyvalley.io/opnsense/FreeBSD:12:amd64/OpenSSL/repo/packagesite.txz: Not Found
Unable to update repository SunnyValley
Error updating repositories!

[mb]

Hi @actionhenkt,

I guess you're installing from getsensei script.

Try this:

Code: [Select]

rm -f /usr/local/etc/pkg/repos/SunnyValley.conf
pkg install os-sunnyvalley-devel
pkg install os-sensei
In the meantime, we'll update getsensei script to handle 20.7

[robsewca]

Not sure if this would be the right place to post, but netmap is reporting this for my virtio interfaces:

Code: [Select]

vtnet0: <VirtIO Networking Adapter> on virtio_pci1
vtnet0: Ethernet address: 00:00:00:00:00:00
vtnet0: netmap queues/slots: TX 4/256, RX 4/128
000.001082 [ 503] vtnet_netmap_attach       vtnet attached txq=4, txd=256 rxq=4, rxd=128
txd and rxd aren't supposed to be the same? This is an OPNsense VM running in KVM (Proxmox). Ring parameters for the interface in the host are set to 2048:

Code: [Select]

Ring parameters for ens4f2:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 2048
RX Mini: 0
RX Jumbo: 0
TX: 2048
And multiqueue is set to 4, which OPNsense correctly applies, but I read in the netmap github that both rxd and txd values must be same to avoid issues. Any thoughts?

[mb]

Hi @robsewca,

It not a big deal. But performance wise, it is advisable that they both have the same number of descriptors.

I do not expect that you see any side effects since normal Internet use is not symmetrical (i.e. download/upload usage is not identical).

But if you do have a fairly symmetrical Internet usage, you'll be limited to the lower descriptor number.

[robsewca]

Thank you @mb,

Any reason why Netmap may be applying two different number of descriptors? Is this something that depends on the hypervisor or OPNsense to decide?

The reason why I'm trying to troubleshoot this is that my internet connection is symmetrical gigabit and while my upload speeds reach 940Mbps, my download speeds are capped at 700-900Mbps, something definitely seems wrong. All hardware offload settings/features are disabled for both OPNsense and the host, but I can't find a way to force higher txd/rxd parameters.

[mb]

Hi @robsewca

Actually netmap does not mangle with interface rx/tx descriptors; instead it inherits the values from the OS.

As you suspected lower download speeds are related to lower RX count.

I wouldn't be surprised if hypervisor would automatically adjust guest vtnet rx descriptors if you also increased the host interface RX descriptor sizes (i.e. set both RX/TX to 4096).

[bunchofreeds]

Hi,

I need PPPoE on my WAN which has known issue's for running IPS. I believe this is due to FreeBSD and its NetMap support of virtual interfaces like PPPoE.
Also my OPNsense is virtual on Proxmox using VirtIO network interfaces.

With a move to HardenedBSD 12.1 and Suricata 5, can I expect to see some change now, or could me testing this help to get this resolved?

Thanks

An earlier post of mine under IPS https://forum.opnsense.org/index.php?topic=16462.0

I have installed a virtual instance of 20.7 although it is not terminating PPPoE currently. I can switch this over if the above testing scope is possible. Install was really easy by the way and has updated and performing quite well. IPS enabled and operational on WAN (although not PPPoE)
   

[mb]

Hi @bunchofreeds,

PPPoE runs over Ethernet. So I see no reason why it should not work provided that Ethernet interface used for the PPPoE connection has no issues with netmap.

Let's have a look at this. Can you do a PPPoE configuration on the latest OPNsense 20.7 beta (kernel 12.1-p5), and try Suricata on it.

Please let me know if this works or not. If it does not work, provide an ifconfig -a output (you can PM me).

[mb]

Ok,

Below file keeps the last status for the Ethernet Drivers <-> netmap compatibility.

https://docs.google.com/spreadsheets/d/1RVj8K3XOzWi-Bkjq6hUxWudu7Cxd8FFTqjLiBMzZWEM/edit#gid=0

This page also explains how you can easily test OPNsense 20.7 netmap.

Feel free to grab a driver, test and provide test results. You should be able to leave comments on the Google Sheets file.

It looks like there's some work to do. We're here to fix.

Just test and provide feedback.

[bunchofreeds]

Thanks for looking into this @mb

I have put the 20.7 proxmox virtual instance of OPNsense inline with PPPoE by backing up my 20.1 config and restoring to 20.7. Really easy to do and was up and running quickly.

Suricata runs successfully and alerts on the LAN interface (Virtio)
Switched to run on the WAN interface and am not receiving alerts. The new log view with v5 Suricata is different and seems to cycle with this when trying to establish IPS on the PPPoE WAN

   Counter | TM Name | Value
    ------------------------------------------------------------------------------------
    Date: 6/14/2020 -- 08:49:09 (uptime: 0d, 00h 08m 17s)
    ------------------------------------------------------------------------------------
    flow.memuse | Total | 7154304
    tcp.reassembly_memuse | Total | 196608
    tcp.memuse | Total | 1146880
    flow_mgr.rows_skipped | Total | 65536
    flow_mgr.rows_checked | Total | 65536
    flow.spare | Total | 10000
    ------------------------------------------------------------------------------------
    Counter | TM Name | Value
    ------------------------------------------------------------------------------------
    Date: 6/14/2020 -- 08:49:01 (uptime: 0d, 00h 08m 09s)
    ------------------------------------------------------------------------------------
    flow.memuse | Total | 7154304
    tcp.reassembly_memuse | Total | 196608
    tcp.memuse | Total | 1146880
    flow_mgr.rows_skipped | Total | 65536
    flow_mgr.rows_checked | Total | 65536
    flow.spare | Total | 10000
    ------------------------------------------------------------------------------------

I am getting the ifconfig -a to you via PM

Here is the log when it successfully runs on the LAN interface

    flow.memuse | Total | 7177096
    tcp.reassembly_memuse | Total | 231424
    tcp.memuse | Total | 1146880
    flow_mgr.rows_maxlen | Total | 1
    flow_mgr.rows_skipped | Total | 65534
    flow_mgr.rows_checked | Total | 65536
    flow_mgr.flows_notimeout | Total | 2
    flow_mgr.flows_checked | Total | 2
    flow.spare | Total | 10000
    flow_mgr.new_pruned | Total | 9
    app_layer.flow.failed_udp | Total | 40
    app_layer.tx.dns_udp | Total | 20
    app_layer.flow.dns_udp | Total | 6
    app_layer.flow.failed_tcp | Total | 4
    app_layer.tx.dhcp | Total | 2
    app_layer.flow.dhcp | Total | 1
    app_layer.flow.tls | Total | 3
    tcp.overlap | Total | 1
    tcp.rst | Total | 16
    tcp.synack | Total | 9
    tcp.syn | Total | 9
    tcp.sessions | Total | 9
    flow.udp | Total | 47
    flow.tcp | Total | 39
    decoder.max_pkt_size | Total | 1514
    decoder.avg_pkt_size | Total | 474
    decoder.udp | Total | 320
    decoder.tcp | Total | 988
    decoder.ethernet | Total | 1594
    decoder.ipv6 | Total | 2
    decoder.ipv4 | Total | 1306
    decoder.bytes | Total | 756423
    decoder.pkts | Total | 1594
    capture.kernel_packets | Total | 1594
    ------------------------------------------------------------------------------------
    Counter | TM Name | Value
    ------------------------------------------------------------------------------------
    Date: 6/14/2020 -- 09:10:57 (uptime: 0d, 00h 01m 52s)
    -----------------------------------------------------------------------------------

[mb]

Hi @bunchofreeds - thanks for the ifconfig output.

I must have guessed it: pppoe is a tun(4) interface. It does not work with netmap(4).

This was the bad news.

Good news is, tun(4) support along with bridge(4) and lagg(4) netmap support in our current scope. Having fixed bugs, this is the first item we'll implement.

[bunchofreeds]

@mb thanks and good luck with the bug fixing!

The support of PPPoE with Netmap would be great for me. I could then run IPS on my WAN and Sensei on my LAN.

Please let me know if I can help with testing any fixes.

Oh and 20.7 is running really well on my Proxmox host. Install was smooth and all services appear to be functioning properly.
Just need some qemu guest support now 😉

All the best.

[mb]

@mb thanks and good luck with the bug fixing!
Oh and 20.7 is running really well on my Proxmox host. Install was smooth and all services appear to be functioning properly.
Just need some qemu guest support now 😉

Kudos to the OPNsense team. Our experience is the same.

Work in em(4) and vtnet(4) has been started:

https://docs.google.com/spreadsheets/d/1RVj8K3XOzWi-Bkjq6hUxWudu7Cxd8FFTqjLiBMzZWEM/edit#gid=0

Tests/feedback with other drivers is welcome and will be much appreciated.

[Voodoo]

Will the netmap kernel changes also be available for the coming 20.7 stable release ?

I would like to not install the beta branch. 20.7 stable should be released next week ?

[franco]

No netmap test kernel for after 20.7 planned, but it's likely we start netmap improvement efforts for 21.1 release.

20.7-RC1 will be out today, final 20.7 next week, yep.


Cheers,
Franco