Messages

I just noticed a weird problem, none of the .mil domains can be resolved within the my home network managed by Opnsense running Unbound as forwarder. Examples: www.navy.mil, www.af.mil, etc ....

Quick googling points to some DNSSEC issue with these US military domains. However unchecking the box "Enable DNSSEC support" doesn't change anything for me.

I'm using DNS over TLS with Google dns servers. Tried switching to Cloudflare or Quad9. Tried enabling all of them. Still doesn't work. I also made sure the setting "Use system nameservers" is NOT selected to make DoT work. I'm also using OISD blocklist, but i doubt that is the culprit.

If i use my phone with cellular connection or connect to Mullvad VPN service directly from my laptop, i can query these domains (with dig command) and visit them normally.

Anyone having this weird issue here?

[Only if i had to manually restart lighttpd, i can access WebUI. It's broken again once i reboot Opnsense until i manually restart webserver.]

I followed this tutorial on official documenation: WireGuard Road Warrior Setup

WG road warrior clients has no problem connecting to internal networks, except that they can't access Web GUI. Firewall allowed the connection and i can see that traffic to Opnsense Web GUI is passed in firewall logs.

I also have Web UI listen to 'HomeWireguard' interface in System > Settings > Administration. There is no log in System > Logs File > Web GUI although i had access log enabled.

Only if i had to manually restart lighttpd, i can access WebUI. It's broken again once i reboot Opnsense until i manually restart webserver.

I suspect that lighttpd service tried to bind to 'HomeWireguard' interface before the this interface is active? Does this sound like a bug?

Is there anyway for me to further debug this problem on my end? I'm seeing no web UI log.

Thank for help!

In my test i had Opnsense VM within promox running on i3-13100 and X710-DA2. Inter-VLAN routing traffic reached 18-19 Gbps, while raw speed between VMs in same VLAN reached 29Gbps. Your CPU doesn't have same single-thread performance but it should do much better job than 1.2Gbps.

I used SR-IOV passthrough, VMs and Opnsense each had its own VF. Traffic between VFs aren't constrained by PHY speed of 10Gbps. All your NICs are capable of SR-IOV so this is recommended way instead of whole NIC passthrough or VirtIO.

I'd like to read more about your detailed setup, switch, VLANs ... Something must be wrong.

[ice]

I don't need 25GbE but E810-XXVDA2 card is at same price as X710-DA2 so I'm thinking why not go for the supposedly better NIC.

I'm aware that FreeBSD has ice driver supporting Intel 800 series but i'm wondering if it is stable as Intel NICs has always been with FreeBSD. One person in this forum seems to have problem with Opnsense not recognizing this card.

Thnx for help guys  :)

Aliases section in Services > Unbound DNS > Host Overrides does not show any of my aliases rule:

https://i.postimg.cc/rydZZYVf/screenshot.png

I can see the aliases in my XML config file, and i can verify that they are working by dig command but none of the rules show up on web UI.

I can even blindly add new aliases via the form but that also allow dupulicated alias and thus creating the warnings PTR record ... already exist on the logs.

I'm planning on acquiring or building new hypervisor host in next year or two, virtualized Opnsense with SR-IOV passthrough will be one of the guests. It appears to me that many servers and motherboards being offered right now have built-in Broadcom SFP+/SFP28 NIC.

I'm aware that Intel NICs are still best, but i would have to buy additional NIC and leave built in ports unused.

So, are the drivers for Broadcom NIC are 'good enough' for Opnsense at the moment or in next couple of years? Specifically, i'm referring to BCM5741x series.

Thanks for any insight  :)

I'm not sure if i'm understanding documentation correctly here:

https://docs.opnsense.org/manual/unbound.html#advanced-configurations

Some installations require configuration settings that are not accessible in the UI. To support these, individual configuration files with a .conf extension can be put into the /usr/local/etc/unbound.opnsense.d directory. These files will be automatically included by the UI generated configuration. Multiple configuration files can be placed there.

But Opnsense config file downloaded from System > Configuration > Backups does not have this custom .conf file i placed in that folder.

I had very same setup with another router without problem but for some unknown reason when i recreate settings with this machine this issue popped up.

I followed Wireguard selective routing with external endpoint in Opnsense documentation, using both Ipv4 and 6. IPv6 gateway monitoring has no problem but IPv4 gateway is always down despite that tunnel is up AND selective routing still works. I tried different monitoring IPs.



Logs are filled with error code 93:



Tunnel settings:




Gateway settings:



I tried comparing this setup with previous one but can't find any difference that could cause this issue. Old router has i210 NICs, this new on has i226.

I tried difference sizes of ICMP payload: 0, 1, 60.

I don't think 23.7 is problem because is has been going on before that.

Would love to have some ideas here to diagnosis this annoying issue. Thanks!

In light of the N100 chip issue with 23.1....

Can you talk more about this issue.

Looks to me that step 9 is for routing traffic from WG address to WG network via WG gateway. IMO it's unnecessary because such route will be auto created when we configure a tunnel gateway nowadays.
For example, this is an excerpt from my routing table:

That's not what I'm seeing, this is all WireGuard routing info when following the guide, including disabling routes and adding a gateway instead, which is needed for selective routing:

Code Select Expand

10.64.0.1          10.66.165.1        UGHS        wg1
10.66.165.1        link#19            UHS         wg1
10.66.165.89       link#19            UH          lo0

10.66.165.89/32 = interface address as supplied by external VPN provider
10.66.165.1 = IP of VPN gateway created through System - Gateways - Single, for selective routing purposes
10.64.0.1 = next hop for VPN connections for gateway monitoring purposes (the Monitor IP set on above gateway))

What did you do that caused the /31 route to appear?

If i stick to the guide on documentation, gateway IP would be 10.66.165.88 in your case (Tunnel address minus 1). Also Far gateway checkbox must be ticked otherwise that gateway IP cannot be set because it's out of range. It appears that this Gateway IP can be pretty much anything and OPNsense guide chose minus 1 tip for convenience.

I don't have 10.66.165.1 or 10.64.0.1 in the route table of Opnsense, but i have 10.66.165.88 because i followed that minus 1 tip.

I don't have that step 9 and selective routing is working, that's why i created this post for more information. I  suspected that step was actually useful.

Have you tried 10.64.0.1 as Gateway IP and tunnel address as 10.66.165.89/10? This allows us to untick the Far gateway option.

The short answer is: not yet.

The DCO kernel mode driver was pulled from FreeBSD 13 (for very good reasons) which OPNsense is built on.

A new driver implementation has since been merged with FreeBSD 14, but that is yet to be released.

As for when OPNsense will begin using the FreeBSD 14 source, that is not yet published on the roadmap, but I would suggest it is some time away...

It appears we won't get DCO until 24.1 at earliest, there is nothing about FreeBSD 14 in that roadmap.

Guys, can we please stay on the topic? I don't care what Juniper or what VyOS does!

I am wondering, is 10Gbit achievable with OPNsense and Chelsio?
If yes, how?
If no, what NICs do?

Is that 6Gbps result done with single stream or multiple stream of iperf3?

I doubt the NIC is bottleneck. You can try turning pf off, but it would mean that a no firewall or ACL on any interface.

This is incorrect in so many ways... Have you heard of Juniper routers or firewalls? They use FreeBSD. Juniper is not the only one there are many high performance network routers that use FreeBSD, Force10 and Extreme comes to mind.

Junos OS Evolved is now Linux-based, they are moving away. Force10's FTOS 10 is also Linux now under the name Dell Networking OS. Also these OSes run on networking gears equipped with ASIC or FPGA to boost performance, so we can't say anything by pointing at these companies.

FreeBSD is far more stable than say a linux distro... Most edge testings of 100GigE+ are done on FreeBSD as it no longer has a software limitation as much as the hardware itself.

Linux Kernel does not have this issue which this cannot be said on various distros. :D

here you go:

https://netflixtechblog.com/serving-100-gbps-from-an-open-connect-appliance-cdb51dda3b99

Linux can also be very stable, depending on the kernel you choose. In that Netflix case study, FreeBSD was used as file server and not router or firewall. The problem presented in that post was not the networking stack itself but feeding the data to networking stack of FreeBSD.

In context of pure software routing/firewalling, this is the contest between Linux's iptables/nftables and FreeBSD's pf. It's no secret that nftables is not only faster but also scales better with number of cores:

https://matteocroce.medium.com/linux-and-freebsd-networking-cbadcdb15ddd

And then there's this new toy called eBPF which is used by Google, Cloudflare, Netflix, Alibaba .... for packet processing. The developments around Linux is just much more active and it's big reason for transition.

Solved by upgrading to 23.1.10_1 hot fix. Something messed up. Should've waited for AA.x.y_z, Got burned by AA.x.y few times already. Really wish for these rolling updates to be more stable.

I was trying to add simple rule for VLAN5:

Action: Pass
Quick: Checked
Interface: VLAN5
Direction: In
TCP/IP version: IPv4
Protocol: TCP/UDP
Source: VLAN5 net
Destination invert: checked
Destination: RFC1918_net

RFC1918_net is alias of networks: 192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12

Got the following rec message:

The following input errors were detected:

    The field Destination bit count is required.
    A valid destination bit count must be specified.

I had same rule in another OPNsense router. But now i can't add any rule with Destination aliases. What's wrong here? Does 23.1.10 has a bug i'm not aware of? How can i roll back to 23.1.9.