OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • Archive »
  • 20.7 Legacy Series »
  • Call for testing: netmap on 20.7
« previous next »
  • Print
Pages: 1 ... 5 6 [7] 8 9 ... 14

Author Topic: Call for testing: netmap on 20.7  (Read 49936 times)

madj42

  • Newbie
  • *
  • Posts: 47
  • Karma: 3
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #90 on: August 10, 2020, 07:57:13 pm »
Just reporting that I've installed the last test kernel and enabled IPS on my PPPoE connection with IGB drivers.  All appears to be good and haven't had issues over the weekend.
Logged

mb

  • Hero Member
  • *****
  • Posts: 912
  • Karma: 97
    • View Profile
    • Sunny Valley Networks
Re: Call for testing: netmap on 20.7
« Reply #91 on: August 10, 2020, 08:25:10 pm »
@madj42, that's great news!. Thanks for letting us know.
Logged

binaryanomaly

  • Full Member
  • ***
  • Posts: 139
  • Karma: 9
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #92 on: August 10, 2020, 09:42:24 pm »
Quote from: mb on August 10, 2020, 06:25:02 pm
@binaryanomaly, thanks for the feedback. vtnet seems to be doing even better than 20.1.

I can definitely confirm that, had lots of page faults already on 20.1. Running smooth and steady since 3 days - great job, well done!
Logged

bunchofreeds

  • Full Member
  • ***
  • Posts: 171
  • Karma: 10
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #93 on: August 10, 2020, 10:45:10 pm »
Running Proxmox with vtnet drivers on WAN (PPPoE) and LAN

Kernel
12.1-RELEASE-p7-HBSD FreeBSD 12.1-RELEASE-p7-HBSD #3  5742b25c4(master)-dirty: Thu Aug  6 16:17:42 PDT 2020     root@sunnyvalley12.localdomain:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

Sensei is running successfully on LAN

I'm unsure if IPS is correctly running on WAN (PPPoE)
Have enabled and set IPS and Promiscuous (Have not selected Syslog or Eve)

Log Output via GUI is:

2020-08-11T08:25:46   suricata[15476]: [100117] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2020-08-11T08:25:46   suricata[15476]: [101288] <Notice> -- opened netmap:pppoe1/T from pppoe1: 0x51b7a149300
2020-08-11T08:25:46   suricata[15476]: [101288] <Notice> -- opened netmap:pppoe1^ from pppoe1^: 0x51b7a149000
2020-08-11T08:25:46   suricata[15476]: [100335] <Notice> -- opened netmap:pppoe1^ from pppoe1^: 0x51b78319300
2020-08-11T08:25:46   suricata[15476]: [100335] <Notice> -- opened netmap:pppoe1/R from pppoe1: 0x51b78319000
2020-08-11T08:24:50   suricata[15476]: [100117] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'is_proto_irc' is checked but not set. Checked in 2002029 and 1 other sigs
2020-08-11T08:24:50   suricata[15476]: [100117] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.gadu.loggedin' is checked but not set. Checked in 2807836 and 0 other sigs
2020-08-11T08:24:50   suricata[15476]: [100117] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.eduphish' is checked but not set. Checked in 2025114 and 0 other sigs
2020-08-11T08:24:30   suricata[89429]: [100183] <Notice> -- This is Suricata version 5.0.3 RELEASE running in SYSTEM mode

However it does not appear to block/drop P2P traffic as a test.
Downloading ubuntu via bit torrent
Ruleset set to drop ET telemetry/emerging-p2p

My Log seems to be broken into two parts. One being the original style logs as shown above. These always appear at the top of the log view.
The second a more like 'stats' that cycles continuously every few seconds. There are LOTS of these being generated.

   flow.memuse | Total | 7154304
    tcp.reassembly_memuse | Total | 196608
    tcp.memuse | Total | 1146880
    flow_mgr.rows_skipped | Total | 65536
    flow_mgr.rows_checked | Total | 65536
    flow.spare | Total | 10000
    ------------------------------------------------------------------------------------
    Counter | TM Name | Value
    ------------------------------------------------------------------------------------
    Date: 8/11/2020 -- 08:29:14 (uptime: 0d, 00h 04m 43s)
    ------------------------------------------------------------------------------------

I do not think that Suricata is operational on my WAN (PPPoE) interface, although it has enabled without errors.

Any ideas how to test this further?




Logged

mb

  • Hero Member
  • *****
  • Posts: 912
  • Karma: 97
    • View Profile
    • Sunny Valley Networks
Re: Call for testing: netmap on 20.7
« Reply #94 on: August 11, 2020, 06:39:39 am »
Quote from: bunchofreeds on August 10, 2020, 10:45:10 pm
I do not think that Suricata is operational on my WAN (PPPoE) interface, although it has enabled without errors.

Hmm, tun(4) implementation is adding an ethernet header (since tun does not have); and mac addresses are all zeros. That might be creating a problem.

Let us have a look deeper; and I'll post updates.

Logged

lewald

  • Full Member
  • ***
  • Posts: 197
  • Karma: 13
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #95 on: August 11, 2020, 04:42:40 pm »
@mb

@lewald, that's great to hear, though I wouldn't expect netmap work might have contributed to the vpn speed. It could be virtio that you can use it with 8 queues.

Maybee. But without the Testkernel i have a lot dequeues on netmap. Now this deueues are gone.
Logged

binaryanomaly

  • Full Member
  • ***
  • Posts: 139
  • Karma: 9
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #96 on: August 13, 2020, 06:07:25 pm »
@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?
Logged

Archanfel80

  • Jr. Member
  • **
  • Posts: 54
  • Karma: 6
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #97 on: August 13, 2020, 08:42:09 pm »
Quote from: binaryanomaly on August 13, 2020, 06:07:25 pm
@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?

No, read the changelog. That is not fix the netmap issues.
Logged

binaryanomaly

  • Full Member
  • ***
  • Posts: 139
  • Karma: 9
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #98 on: August 13, 2020, 08:56:59 pm »
Quote from: Archanfel80 on August 13, 2020, 08:42:09 pm
Quote from: binaryanomaly on August 13, 2020, 06:07:25 pm
@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?

No, read the changelog. That is not fix the netmap issues.

"From the reported issues we still have more logging quirks to investigate and especially Netmap support (used in IPS and Sensei) is lacking in some areas that were previously working. Patches are being worked on already so we shall get there soon enough. Stay tuned."

That could mean anything...
Logged

FullyBorked

  • Sr. Member
  • ****
  • Posts: 258
  • Karma: 22
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #99 on: August 13, 2020, 08:57:24 pm »
Quote from: Archanfel80 on August 13, 2020, 08:42:09 pm
Quote from: binaryanomaly on August 13, 2020, 06:07:25 pm
@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?

No, read the changelog. That is not fix the netmap issues.

To be a little fair the change log opening sentence seems a bit ambiguous.  I don't think it was intended that way but the first time I read it I thought it was saying there were other changes that weren't noted in the log.

Quote
Small update here with security advisories, multicast fixes and logging reliability patches amongst others.

Also I think he is asking if it will revert his changes, not if it has been fixed in the release. 
Logged

franco

  • Administrator
  • Hero Member
  • *****
  • Posts: 13679
  • Karma: 1176
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #100 on: August 13, 2020, 09:03:45 pm »
"amongst others" references the full change log below. It's intentionally ambiguous in the sense that the actual changes are listed below. If you don't see your issue there it's probably just that.

The second paragraph is more loose in terms of content from release to release. It is meant to hint at past and future events. In this case it unambiguously states that Sensei and IPS issues are not yet resolved in the release.

I'm not sure how to make this any clearer other than: don't panic and use 20.1 if you must. ;)


Cheers,
Franco
Logged

FullyBorked

  • Sr. Member
  • ****
  • Posts: 258
  • Karma: 22
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #101 on: August 13, 2020, 09:05:39 pm »
Quote from: franco on August 13, 2020, 09:03:45 pm
"amongst others" references the full change log below. It's intentionally ambiguous in the sense that the actual changes are listed below. If you don't see your issue there it's probably just that.

The second paragraph is more loose in terms of content from release to release. It is meant to hint at past and future events. In this case it unambiguously states that Sensei and IPS issues are not yet resolved in the release.

I'm not sure how to make this any clearer other than: don't panic and use 20.1 if you must. ;)


Cheers,
Franco

No panic here :)  I just read it wrong, initially.  Thought maybe someone else did too.  No need to rewrite anything at all.  I'm patiently waiting on 20.7 and figure it'll work when it works and it will be great as usual. 
Logged

franco

  • Administrator
  • Hero Member
  • *****
  • Posts: 13679
  • Karma: 1176
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #102 on: August 13, 2020, 09:07:39 pm »
Thanks, next week will have progress on this front for sure. If we can confirm quickly we may even start adding patches to 20.7.2 already. Just want to keep the changes small and impact big... in the positive sense of course.


Cheers,
Franco
Logged

binaryanomaly

  • Full Member
  • ***
  • Posts: 139
  • Karma: 9
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #103 on: August 13, 2020, 11:01:54 pm »
Quote from: franco on August 13, 2020, 09:03:45 pm
I'm not sure how to make this any clearer other than: don't panic and use 20.1 if you must. ;)

vtnet page faults on anything besides the test kernel from mb here. So I'll stay with the test kernel until the fix finds its way upstream :)
Logged

heresjody

  • Newbie
  • *
  • Posts: 25
  • Karma: 2
    • View Profile
Re: Call for testing: netmap on 20.7
« Reply #104 on: August 14, 2020, 06:06:35 am »
Quote from: mb on August 06, 2020, 04:25:14 pm
@sorano, it does not seem to be related.

Please follow below steps and see if this kernel is of help:

Code: [Select]
[root@20gw /root]# cd /boot/
[root@20gw:/boot # fetch https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/kernel-12.1-0805-2.tar.gz
kernel-12.1-0805-2.tar.gz                           45 MB 4980 kBps    10s
[root@20gw /boot]# mv kernel kernel.stock.save
[root@20gw /boot]# tar zxf kernel-12.1-0805-2.tar.gz 
[root@20gw /boot]# reboot

After the reboot, you should be able to see this kernel information:

Code: [Select]
root@20gw:~ # uname -a
FreeBSD 20gw.local 12.1-RELEASE-p7-HBSD FreeBSD 12.1-RELEASE-p7-HBSD #2  5742b25c4(master)-dirty: Wed Aug  5 22:20:24 PDT 2020     root@sunnyvalley12.localdomain:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64
root@20gw:~ #

To restore stock OPNsense kernel:

Code: [Select]
# cd /boot
# rm -rf kernel
# mv kernel.stock.save kernel
# reboot

Is it possible to post a test kernel based on the 20.7.1 kernel? Then I can test Surricata on PPPoE WAN without my girlfriend complaining about not being able to watch IPTV  ::)
Logged

  • Print
Pages: 1 ... 5 6 [7] 8 9 ... 14
« previous next »
  • OPNsense Forum »
  • Archive »
  • 20.7 Legacy Series »
  • Call for testing: netmap on 20.7
 

OPNsense is an OSS project © Deciso B.V. 2015 - 2023 All rights reserved
  • SMF 2.0.19 | SMF © 2021, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2