Messages

1

20.7 Legacy Series / inconsistent tunables?

« on: December 27, 2020, 01:14:10 pm »

Hi all,
I just noticed that the default settings for tunables do not seem to match:

net.inet.ip.redirect = 0

but

net.inet6.ip6.redirect = 1


Is there a reason for disabling redirects for ipv4 but not ipv6? Or are the tunables similar only in name but not function (which would be bad, too, I guess.)

Thanks,

Martin

2

20.7 Legacy Series / Re: Traffic shaper and ACK packets

« on: May 25, 2020, 07:42:50 pm »

The current code:

Code: [Select]

tcpflags ack
Ad suggested:

Code: [Select]

tcpflags ack,!pshwhich I think would be an improvement.

My suggestion would be to try and narrow it down to packets doing nothing but ack by excluding larger payloads. E.g.:

Code: [Select]

tcpflags ack iplen 52
Shall I open an issue for this?

3

20.7 Legacy Series / Re: Traffic shaper and ACK packets

« on: May 25, 2020, 04:02:28 pm »

Very OT:

There is this one (weirder than average) episode of X-Files where Mulder meets a genie and actually has three wishes.
His first one is: peace on earth
after which he is the only living human being left on the planet.
So, yes. You should be specific when you ask for something. ;-)

4

20.7 Legacy Series / Re: Traffic shaper and ACK packets

« on: May 24, 2020, 03:10:03 pm »

Thanks to both of you for your replies.

@franco I agree that there is currently no discrepancy w/r to the documentation or the behavior. My point was that the functionality as it is defined right now is not particularily useful.

@AdSchellevis I guess my hope would be that eventually the shaper rule editor will allow for a more complete subset of the choices that the FW rule editor offers.
That would not only be true for tcp flags, but also specifying the type of icmp packet instead of picking them all, etc.

But certainly, all of this is just 'nice to have'.

Martin

5

20.7 Legacy Series / Traffic shaper and ACK packets

« on: May 23, 2020, 11:43:11 am »

Hi there,

I noticed something weird when trying to prioritize ACK packages.
Selecting "tcp (ACK packets only)" in the proto drop-down results in almost all tcp traffic being matched.
Doing something similar in m0n0wall or even the firewall solution that shall not be named, did not result in comparable behavior.
Though, with those solutions I could (and did) specify the packet size to something very small so only empty ACKs where prioritized. That does not seem to be an option in the OPNsense shaper currently?

If "tcp (ACK packets only)" matches any packet having the ACK flag set, then that is not (to me at least) particularly useful...

6

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: March 13, 2020, 08:40:03 am »

That's what I had attempted. The problem is, adding ng_bridge on boot is not sufficient (though that would have seemed like the obvious thing).
I have not yet made out what other module is needed. :-(

7

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: March 09, 2020, 06:03:06 pm »

Franco, let's forget about the unsure stuff for a moment (For the unsure stuff I will have to reinstall 19.7 on some box and test it.).
What I am certain about:

Say you create a bridge, add ports and assign the bridge as an interface e.g. OPT1.
And then enable flowd for OPT1.

Pre 20.x it would record traffic out of the box. With 20.x it does not.

This is definitely due to the modules mentioned and is what I can get back working when adding the modules back in.
The previous post explained how I was so far unable to narrow it down to which specific modules are responsible.

8

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: March 08, 2020, 11:31:28 am »

Thanks, Ad.

I have tried to find out what the missing modules might be, but so far, no luck.

If I do not touch modules, I end up with the graph as shown in the attached vanilla.png
and the output of

Code: [Select]

ngctl types is:

Code: [Select]

There are 10 total types:
      Type name   Number of living nodes
      ---------   ----------------------
        ksocket       1
        netflow       1
         tcpmss       1
          pppoe       1
          ether      10
            tee       1
            ppp       1
          iface       1
           mppc       0
         socket       6

If I put all these (https://github.com/opnsense/core/blob/stable/19.7/src/etc/rc.loader.d/20-netgraph) into /boot/loader.conf.local, I end up with the graph as shown in the attached modules.png and this output

Code: [Select]

There are 31 total types:
      Type name   Number of living nodes
      ---------   ----------------------
        netflow       4
         socket       6
           vlan       0
            vjc       0
            tty       0
            tee       1
         tcpmss       1
          ether      13
         eiface       0
        rfc1490       0
          pred1       0
           echo       0
        pptpgre       0
          pppoe       1
        deflate       0
            ppp       1
          async       0
           pipe       0
          cisco       0
       one2many       0
           mppc       0
            car       0
         bridge       0
            lmi       0
           l2tp       0
        ksocket       4
            bpf       0
          iface       1
             UI       0
           hole       0
    frame_relay       0

I assumed I would see which types are missing by listing the used node types, but apparently, that is not true (or there might be ng_xyz modules that are not node types themselves, but add functionality to existing types?).

Also note that, apparently, I have to reboot after adding modules. Using

Code: [Select]

kldload ng_xyz to load each of the modules mentioned in https://github.com/opnsense/core/blob/stable/19.7/src/etc/rc.loader.d/20-netgraph without a reboot and then doing

Code: [Select]

/usr/local/etc/rc.d/netflow restart, I do not get the same results.

How would I go about narrowing this down without doing trial and error and module by module and having to reboot? Is there another service that needs restarting or is a reboot really necessary?

9

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: March 07, 2020, 09:58:30 am »

Giving this a bump as it is still the same behavior in OPNsense 20.1.2.

I can get some of the interfaces to log netflow data by loading kernel modules that are no longer loaded automatically since 20.1.x.

But I have not been able to get my pppoe or openvpn WAN ports to log egress traffic. (I have to admin though, that I do not know for certain these two ever did).

At any rate, manually adding kernel modules to be loaded on boot in order to get built-in features (Netflow) to work seems like a band-aid to me. :-) Are there plans to overhaul the reporting section?

10

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: February 04, 2020, 02:35:13 pm »

Thanks, franco

Putting the content of this file https://github.com/opnsense/core/blob/stable/19.7/src/etc/rc.loader.d/20-netgraph
into /boot/loader.conf.local brought some improvement:

Code: [Select]

root@gw01:~ # /usr/local/etc/rc.d/netflow restart
setup bridge1
setup bridge0
setup bridge2
setup ovpnc1 [egress only]
ngctl: send msg: No such file or directory
error ovpnc1: cannot create netflow node for ovpnc1
setup pppoe0 [egress only]
ngctl: send msg: No such file or directory
error pppoe0: cannot create netflow node for pppoe0

So, some of the necessary modules are now loaded that weren't before.
At least, Reporting ->Netflow -> Cache now lists the bridges and their counters.

Obviously, however, my egress interfaces still don't collect data.

11

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: February 04, 2020, 11:05:00 am »

Thank you for explaining the flow of things.

The result of /usr/local/etc/rc.d/netflow restart

Code: [Select]

root@gw01:~ # /usr/local/etc/rc.d/netflow restart
setup bridge1
ngctl: send msg: No such file or directory
error bridge1: cannot create netflow node for bridge1
setup bridge0
ngctl: send msg: No such file or directory
error bridge0: cannot create netflow node for bridge0
setup bridge2
ngctl: send msg: No such file or directory
error bridge2: cannot create netflow node for bridge2
setup ovpnc1 [egress only]
ngctl: send msg: No such file or directory
error ovpnc1: cannot create netflow node for ovpnc1
setup pppoe0 [egress only]
ngctl: send msg: No such file or directory
error pppoe0: cannot create netflow node for pppoe0

I am reading manpages to try and understand the syntax, etc., but for the record:
With pppoe0 device, for example, /usr/local/etc/rc.d/netflow restart tries to do:

Code: [Select]

root@gw01:~ # /usr/sbin/ngctl shutdown netflow_pppoe0
ngctl: shutdown: No such file or directory
root@gw01:~ # /usr/sbin/ngctl mkpeer pppoe0: netflow lower iface19
ngctl: send msg: No such file or directory


12

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: February 04, 2020, 09:08:29 am »

Actually, no. There are no entries on that page at all.
What process should be generating those?

And do I understand the process correctly:
XYZ generates the counters, they are picked up by flowd and put into the /var/log/flowd.log and then the aggregate script writes it into the sqlite DBs in /var/netflow?

I would really like to see a chart for this process, btw. :-)

13

20.1 Legacy Series / Re: flowd not working after upgrade.

« on: February 03, 2020, 07:28:58 pm »

Is there any other piece of information I could supply that might help diagnose this?

Even after the patch mentioned by AdSchellevis, followed by a reboot, flowd.log does not get written to. At all.

14

20.1 Legacy Series / Re: High memory usage

« on: February 02, 2020, 11:30:20 pm »

Decidedly odd.
I just setup a new VM on my Proxmox cluster and with 8 GB RAM, OPNsense (20.1) showed only a usage of a bit under 300M.

Can you share more information on the setup of the VM?

15

20.1 Legacy Series / Re: High memory usage

« on: February 02, 2020, 08:33:24 pm »

A clean OPNsense install has really low memory usage in my experience.

Where did you take the information from? About the RAM usage, I mean? From the VM status in Proxmox?

Because: Outside of the VM, Proxmox (KVM) does not reliably / accurately display memory usage inside FreeBSD hosts.

FreeBSD's memory management knows several memory classes: Active, Inactive, Laundry, Free, Wired
(See https://wiki.freebsd.org/Memory)

KVM only displays the amount of 'used' memory. I do not know how they calculate that.
In short: I would disregard the value KVM displays and go with OPNsense's dashboard info.