Messages

Thanks, looks like they fixed that  ;)

(Spike is the reboot after changing the setting)

Hi,

I run OPNsense on Proxmox. With suricata enabled I max out at around 3 Gbps when testing, effectively it seems to be around 1.3 Gbps.

Looking for bottlenecks I discovered that
1. OPNsense reports a CPU usage of about 40%
2. Proxmox reports a CPU usage of >80%

This is a bit odd since one must be wrong. Proxmox would indicate that CPU is maxed out whereas according to OPNsense there's plenty of CPU left.

I assume Proxmox reports the real figure. Why does OPNsense say something completly different?

Habe hier seit Jahren unbound auf opnsense am laufen (ohne blocklists), da unbound so auch grad die dhcp IPs handhaben kann.

Für die clients habe ich pihole in einem lxc container, der unbound als resolver nutzt und alles andere geblockt bzw. port 53 geNATted auf pihole.

Funktioniert für mich sehr gut. Hatte auch mal adguard getestet aber pihole stimmt für mich besser. Der lxc container macht mir keinen Aufwand da alles auf proxmox läuft.

Thanks for your feedback.

I just found out that in fact "URL Tables (IPs)" work with lists of fqdns as well such as https://gist.githubusercontent.com/ckuethe/f71185f604be9cde370e702aa179fc2e/raw/53fe52046836ac3009e9505b7b8b8b5de42d84e3/doh-blocklist.txt

Nice - It's a feature not a bug 😉

Maybe someone should update the docs 😎


PS: Using pfctl or editing /usr/local/etc/filter_tables.conf seems not to remain persistent.

Interestingly this works from cli:

Code Select Expand


sudo pfctl -t H_DoG -T add one.one.one.one doh.dns.sb dnsforge.de dns.google dns.google.com doh.dns.apple.com doh.seby.io dns-nyc.aaflalo.me ibksturm.synology.me fi.doh.dns.snopyta.org doh.cleanbrowsing.org doh.tiarap.org jp.tiarap.org doh.powerdns.org dns.switch.ch digitale-gesellschaft.ch resolver-eu.lelux.fi doh.li dns.aa.net.uk dns.adguard.com dns-family.adguard.com cloudflare-dns.com mozilla.cloudflare-dns.com family.cloudflare-dns.com security.cloudflare-dns.com doh-de.blahdns.com doh-fi.blahdns.com doh-jp.blahdns.com doh.eastus.pi-dns.com doh.westus.pi-dns.com doh.northeu.pi-dns.com doh.centraleu.pi-dns.com doh.familyshield.opendns.com doh.opendns.com doh.dnslify.com doh.xfinity.com dns.rubyfish.cn captnemo.in doh.captnemo.in dns.nextdns.io doh-2.seby.io doh.tiar.app jp.tiar.app doh.42l.fr doh.libredns.gr dns.flatuslifir.is dns10.quad9.net dns11.quad9.net dns9.quad9.net dns.quad9.net dohdot.coxlab.net doh.ffmuc.net ordns.he.net dns.dnsoverhttps.net ibuki.cgnat.net rdns.faelix.net dns.hostux.net applied-privacy.net doh.applied-privacy.net commons.host dns.twnic.tw doh.crypto.sx odvr.nic.cz


Unfortunately I can't use this externally created alias in a fw-rule in the OPNsense UI but technically this should work.
Edit: If I create an alias from within the UI and then populate from cli this does work! 🙌🏻

Which seems to confirm my assumption that only existing capabilities would have to be combined?

Hi,

Wouldn't it be nice if one could provide a list of domains to be blocked such as DoH or adserver domains as an alias to be blocked.

It seems to me that functionally speaking almost everything is there
1. loading remote lists like "URL Tables (IPs)"
2. fqdns can be added as "hosts" for blocking

But just not the combination of the above two and maintaining fqdns manually as aliases is cumbersome.
Wouldn't it be possible to combine both capabilities and allow fetching remote fqdn lists for blocking or am I missing something?

I am aware that similar could be achieved with suricata, proxy or dns blocking but none of them would be as practical and effective as being able using fqdns lists in firewall rules.

Thoughts?

since the 11600 is a 6-core/12-thread cpu I presume '4 cores for opnsense' means you're running virtualised? Are passing the NICs thru to the VM, or you using vmxnet3/virtio?

I run it virtualized on promox. I am currently using bridged virtio NICs but have also tested passthrough but the difference was not significant if I recall correctly. I will set it up in passthrough again, soon.

Regarding the values of the initial post, I realized later that when switching on/off suricata there's a delay until it's fully up so the the 8 Gbps measurement is not accurate it's rather 3 Gbps with Suricata on. I think it's related to single CPU restrictions.

I had a similar problem with 21.7.8, some upgrade to a version before had introduced it suddenly.
I even had to reboot the vmhost to make the OPNsense VM restart it got stuck so bad.
Also KVM and proxmox here but I'm not using qemu guest.

Fortunately the upgrade to 22.1 fixed it.

I was thinking of exporting the config and do a clean setup from scratch in parallel to see if that fixes it.
Maybe that could be an approach for you as well?

The results are pretty good. Constantly 2-3ms e2e.

After digging deeper I now found a

Code Select Expand

rx_no_dma_resources issue on the NIC.
It looks as if this could be the root cause of the intermittent issues I am experiencing. I have no idea why this suddenly appeared, may be related to some kernel upgrade or so on the vmhost itself.

I'm still confused though that OPNsense reports package loss in the Reporting/Health/Quality section but not in System/Gateway/Single.

@Franco: This might be a bug or I am not getting how this is intended to work.

Thanks, indeed there was something wrong with the host resolution, using the IP now.

Also it seems to show some packet loss, I'll have to investigate further.
Thanks for your help so far 👍🏻

Thanks, so pretty standard config.

Code Select Expand


+ Remote

menu = Remote
title = Remote check

++ cloudflare

menu = Cloudflare
title = 1.1.1.1 check

#probe = FPingNormal
host = 1dot1dot1dot1.Cloudflare-dns.com


My graph looks like below, not sure why I'm getting "u" as unit for the y-axis.
Anything of concern?

Code Select Expand

fping -s 1.1.1.1
on command line returns 2.43ms avg which I do not recognize in the graph.

Thanks a lot! Set it up.
Would you mind sharing your config?

Hi,

Since quite a while I experience occasional connection interruptions and can observe packet loss on OPNsense (not just since 22.1). I do suspect my ISP but I have not enough evidence to approach it yet.

I have already activated gateway monitoring. Interestingly packet loss is displayed as 0.0% in System -> Gateways -> Single.
Although Reporting -> Health -> Quality displays packet loss for the Gateway.
Which one is correct?

How can I investigate this further in OPNsense?

Thanks

[any]

Hi,

The rule blocks any connections where the destination is one of the IPs in the blacklist.

You would need another rule to block any connections where the source is one of the IPs in the blacklist.
But most setups do not allow incoming traffic from the WAN interface anyway so this is kinda obsolete.

You're right, the any for the direction may not even be required but I just didn't bother since any works just fine as well.

-b

I wonder why the official documentation doesn't mention this other option too...

I think the official documentation does not get updated too often.
(Updating it is also not as simple and straightforward as with modern wysiwyg wikis.)