Topics

It's been a few years since I last attempted to run opnsense on arm - looks like things haven't changed and opnsense remains to be an x86/amd64 only platform.

Has there been an official stance about aarch64 by the opnsense devs? My nanopi R6S is great with OpenWrt but opnsense is a much better all-in-one platform to run for home firewall.

I'm having MTU issues (unable to load websites - dell remote management) over the IPsec tunnel. I have lowered the MTU and MSS settings on my LAN but still facing issues - if I reboot the opnsense it will work for a few minutes so it seems some traffic may respect MSS but then stops working.

pfsense seems to have special settings under IPsec for this condition per https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/advanced.html

other opnsense users seem to have reported the same issue without resolution: https://forum.opnsense.org/index.php?topic=17881.0

any idea what can be done?

I've been using OPNsense for several years, the product is great and has been rock solid for Cable modem ISPs. Several months ago I switched to a fully wireless 5G home internet ISP (T-mobile) which seems to be causing opnsense to randomly crash and services to stop working.

The pattern I have observed is the following:
- internet on LAN and other VLANs stop working completely
- opnsense Web UI indicates a crash and to report it (not super helpful - but have done several reports already)
- opnsense dashboard shows the following services as stopped or failed:
* Unbound DNS
* flowd_aggregate
* My OpenVPN clients are down
* All my gateway monitors are 'offline'

Sometimes restarting unbound is enough to bring everything back online - but that is only 1 out of 4 times. Most times I have to do a full reboot of opnsense to get things back online. I had tried to do 'reset' WAN interface from the interface status menus to no avail.

While I admit that my setup may be complex since I run opnsense as a VM on proxmox and then WAN connectivity to ISP equipment is connected directly to proxmox host to a bridge interface I use for clients I want to be directly connected to ISP.

Has anyone experienced or is it well known that if a WAN connection may be unstable there is odd behavior in opnsense to be expected?

To be clear, I have noticed the issue to be the ISP equipment (Nokia fastmile 5G router) which even when directly connected to it there are 'brief connection interruptions' of sometimes 30 seconds to a few minutes while the modem reconnects to other LTE/5G cell towers. But I would expect opnsense to be able to recover from this without a reboot - or perhaps I am missing something?

Restarting downed services do not fix this, restarting WAN interface via web UI does not fix it either. opnsense does get an IP but somehow its routing table to the internet just doesn't work until a reboot occurs.

I have two WAN interfaces and looking to setup failover - I have configured an IPv4 and IPv6 gateway per each WAN... so we have a total of 2 ISP/WAN and 4 gateways.

When trying to define "Tier 1" it looks like I can't select Tier 1 more than once - how can I ensure that both IPv4/IPv6 traffic both uses WAN1 until it fails then defaults to WAN2?

Maybe I am confused because each WAN link has a gateway monitor for each IP protocol but I think what I am trying to achieve is simple. I did try to create "a gateway group of another group" but alas that isn't possible lol

Curious if it may be known that wireguard is almost 2x or 3x faster than FreeBSD implementation?

I have been benchmarking wireguard performance to a local datacenter VPS that will become my gateway soon (I have 5G internet at home and CGNAT).

I setup wireguard clients on 3 platforms and did the same tests.

Debian 10 (wireguard kernel mod):
-- downloading data from VPS to home

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.03  sec   453 MBytes   127 Mbits/sec   49             sender
[  5]   0.00-30.00  sec   442 MBytes   124 Mbits/sec                  receiver

-- uploading

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  24.1 MBytes  6.75 Mbits/sec   19             sender
[  5]   0.00-30.05  sec  23.9 MBytes  6.68 Mbits/sec                  receiver



pfsense 2.5.0 CE (built-in wireguard in kernel according to them):

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.25  sec   176 MBytes  48.7 Mbits/sec    1             sender
[  5]   0.00-30.00  sec   170 MBytes  47.5 Mbits/sec                  receiver

iperf Done.
[2.5.0-RELEASE][root@pfSense.pf]/root:
--upload

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  24.6 MBytes  6.89 Mbits/sec   20             sender
[  5]   0.00-30.26  sec  23.7 MBytes  6.56 Mbits/sec                  receiver

opnsense 21.1 (go-wireguard plugin):

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.25  sec   104 MBytes  28.9 Mbits/sec  1860             sender
[  5]   0.00-30.00  sec   101 MBytes  28.4 Mbits/sec                  receiver
-- upload

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  15.4 MBytes  4.32 Mbits/sec   22             sender
[  5]   0.00-30.26  sec  15.1 MBytes  4.19 Mbits/sec                  receiver

iperf3 without tunnel from opnsense:

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.28  sec   352 MBytes  97.5 Mbits/sec  999             sender
[  5]   0.00-30.00  sec   343 MBytes  96.0 Mbits/sec                  receiver
-- upload

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  13.4 MBytes  3.74 Mbits/sec   34             sender
[  5]   0.00-30.26  sec  13.3 MBytes  3.68 Mbits/sec                  receiver

For good measure, speed test from debian 10 box outside tunnel:

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.08  sec   321 MBytes  89.6 Mbits/sec  1278             sender
[  5]   0.00-30.00  sec   311 MBytes  87.1 Mbits/sec                  receiver

-- upload

Code Select Expand

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.00  sec  31.0 MBytes  8.67 Mbits/sec   33             sender
[  5]   0.00-30.04  sec  30.8 MBytes  8.59 Mbits/sec                  receiver


speedtest.net results https://www.speedtest.net/my-result/d/f2d48b53-2bfe-4ed8-b76e-9c2fecdd8137

I really wish that FreeBSD had similar performance to the Debian 10 VM - the setup is identical for all wireguard clients. Same ISP network router is upstream, connecting via IPv6 to the VPS server (due to my ISP being IPv6 only network its best to avoid IPv4 hosts direct connects due to CGNAT). I may end up using this debian box as secondary WAN for opnsense so that I can keep the performance gains for my traffic.

I have been experimenting with BBR congestion algorithm from Google. It's widely available in Linux. Apparently FreeBSD may have had it committed in: https://reviews.freebsd.org/rS352657 (I am not sure how to check)

*edit: https://svnweb.freebsd.org/base?view=revision&revision=352657 * perhaps the feature request for OPNsense is to enable the build flags mentioned here?

This
is a completely separate TCP stack (tcp_bbr.ko) that will be built only if
you add the make options WITH_EXTRA_TCP_STACKS=1 and also include the option
TCPHPTS

In linux I have seen an improved performance with it and wanted to enable it for my OPNsense at home, does anyone have any guidance on how TCP BRR congestion algorithm is enabled for use in the system?

Found this but it isn't relevant to opnsense (https://fasterdata.es.net/host-tuning/freebsd/)

Code Select Expand

[root@fw ~]# ls /boot/kernel/cc_* | grep -v symbols
/boot/kernel/cc_cdg.ko
/boot/kernel/cc_chd.ko
/boot/kernel/cc_cubic.ko
/boot/kernel/cc_dctcp.ko
/boot/kernel/cc_hd.ko
/boot/kernel/cc_htcp.ko
/boot/kernel/cc_vegas.ko


If BRR is news to you, some references:
https://github.com/google/bbr
https://atoonk.medium.com/tcp-bbr-exploring-tcp-congestion-control-84c9c11dc3a9
https://www.cyberciti.biz/cloud-computing/increase-your-linux-server-internet-speed-with-tcp-bbr-congestion-control/

My ISP (5G wireless home internet / T-mobile) gives us a dumb modem that does not allow 'bridge mode' the ISP themselves doesn't do IPv6 prefix delegation. Looking for help fixing issues with http://ipv6-test.com/ and http://test-ipv6.com/ as they fail.

I'm not sure what my options are here other than NAT IPv6? I'm not familiar with IPv6 much but I can connect directly to the modem obtain a V6 address and all dual-stack tests pass.

When opnsense connects to the modem, it obtains a unique IPv6 (non-local-link) and the WAN settings are set for DHCPv6 and my LAN is set "Track interface".

I have been struggling with this for a few weeks - to make sure nothing advanced (vlan, complex rules, vpn settings etc) I ended up spinning up a fresh install of opnsense - straight out of the box with default settings one WAN and one LAN to see if it would work. I did this with both opnsense and pfsense - my luck has been that IPv6 dual stack doesn't work in neither pfsense or opnsense behind this modem.

Perhaps I am missing something to try here? Open to suggestions.

In the alternative - any steps or quick guide on using NAT IPv6 on the LAN and then use outbound NAT to share the single IPv6 I am getting from WAN? or I wonder if pfsense can bind multiple IPv6 addresses to the WAN interface and manage it somehow intelligently by itself and without NAT?

I'm looking to ensure that UDP packets sent outbound to a wireguard server from opnsense are tagged with high TOS priority DSCP 46 (voice).

If memory serves me right - I can modify TOS/DSCP when a rule matches on the firewall BUT I believe OUTBOUND rules is something that opnsense wouldn't be able to handle for when the wireguard server is opnsense itself?

Can someone help validate if the above is accurate - any hints on making this possible? short workaround I can think of is to have another device on the network (not opnsense) be the wireguard client and then have opnsense mark the packet from that client outbound - ideally though opnsense should be able to do this packet mangling as soon as it leaves the wireguard binary if it runs on itself.

I'm playing around with TunSafe (a wireguard fork) which has traffic obfuscation built-in to circumvent ISP throttle or blocks on native wireguard protocol which is not hidden.

Plain wireguard with my ISP will be throttled to 5mbps maximum however when using tunsafe over TCP 443 the traffic is camouflaged as TLS/Web and I am able to hit 100Mbps.

I'm wondering how hard it would be to try to use opnsense webgui to detect the new network interface that compiling and running the binary on the system would create - is this hard?

Info: https://github.com/TunSafe/TunSafe

I'm going to spin a VM opnsense to play with but could use some guidance on basic stuff to do or look for. I already know how to compile and get the tun0 interface up on the system but unsure how to tie things together with the opnsense ecosystem / GUI.

[Problem is that wg0 is where opnsense should be sending those responses and it should outbound NAT 172.20.0.20 as 10.0.0.10. Routing this via wg0 seems to be not happening and instead going thru opnsense-WAN.]

I'm trying to setup Port Forwarding via VPN interface, the packets are being NAT'd correctly but when the response packets are being sent the VPN gateway (wg0) is being ignored and WAN (em0) is used.

Goal:

Network topology looks like:

Internet >> VPS (public IPv4) << wireguard tunnel >> opnsense FW (wg0) << client on DMZ interface

WAN-vps (assume its 1.2.3.4)
wg0 (vps-server IP 10.0.0.1)
wg0 (opnsense wg0 IP 10.0.0.10)
client (dmz IP 172.20.0.20)

My goal is to forward port 32400 for Plex media server.
IP 1.2.3.4:32500 is expected to connect to VPS via public IP, be forwarded via wg0 and arrive at opnsense.
looking to keep original network source TCP/IP addresses to track who is connecting and from where in my logs.

opnsense should NAT incoming wg0 traffic to port 32400 and translate to 172.20.0.20 (DMZ interface host).

opnsense is translating traffic and when client is responding they it is for public IP addresses - I expected by policy based routing rule on DMZ interface to apply here. I have explicitly set the gateway for non-local networks to be using wg0 interface but tcpdump of WAN on opnsense show otherwise. Packets are being sent out of the non-wireguard tunnel.

Any help here would be amazing. Been stuck here.

My test scenario, using: https://www.yougetsignal.com/tools/open-ports/ to check port 32400

On WAN-VPS (wireguard gateway with forwarding enabled - confirmed packet is coming in to Public IPv4):

Code Select Expand

listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
17:54:11.413547 IP 76.175.136.3.50396 > 1.2.3.4.32400: Flags [S], seq 1644582163, win 65535, options [mss 1460,nop,wscale 7,sackOK,TS val 3902988 ecr 0], length 0
17:54:12.944644 IP 198.199.98.246.40123 > 1.2.3.4.32400: Flags [S], seq 3344110111, win 14600, options [mss 1460,sackOK,TS val 729869398 ecr 0,nop,wscale 8], length 0
17:54:13.942881 IP 198.199.98.246.40123 > 1.2.3.4.32400: Flags [S], seq 3344110111, win 14600, options [mss 1460,sackOK,TS val 729869648 ecr 0,nop,wscale 8], length 0
17:54:13.957298 IP 198.199.98.246.40125 > 1.2.3.4.32400: Flags [S], seq 1528187397, win 14600, options [mss 1460,sackOK,TS val 729869651 ecr 0,nop,wscale 8], length 0
17:54:14.954744 IP 198.199.98.246.40125 > 1.2.3.4.32400: Flags [S], seq 1528187397, win 14600, options [mss 1460,sackOK,TS val 729869901 ecr 0,nop,wscale 8], length 0
17:54:14.989967 IP 198.199.98.246.40128 > 1.2.3.4.32400: Flags [S], seq 1176045227, win 14600, options [mss 1460,sackOK,TS val 729869909 ecr 0,nop,wscale 8], length 0
17:54:15.986776 IP 198.199.98.246.40128 > 1.2.3.4.32400: Flags [S], seq 1176045227, win 14600, options [mss 1460,sackOK,TS val 729870159 ecr 0,nop,wscale 8], length 0


From VPS-wg0 interface it shows packet being forwarded with original IPv4 headers as wanted with a destination of 10.0.0.10 the opnsense wg0 interface:

Code Select Expand

17:54:11.413562 IP 76.175.136.3.50396 > 10.0.0.10.32400: Flags [S], seq 1644582163, win 65535, options [mss 1460,nop,wscale 7,sackOK,TS val 3902988 ecr 0], length 0
17:54:12.944663 IP 198.199.98.246.40123 > 10.0.0.10.32400: Flags [S], seq 3344110111, win 14600, options [mss 1460,sackOK,TS val 729869398 ecr 0,nop,wscale 8], length 0
17:54:13.942893 IP 198.199.98.246.40123 > 10.0.0.10.32400: Flags [S], seq 3344110111, win 14600, options [mss 1460,sackOK,TS val 729869648 ecr 0,nop,wscale 8], length 0
17:54:13.957321 IP 198.199.98.246.40125 > 10.0.0.10.32400: Flags [S], seq 1528187397, win 14600, options [mss 1460,sackOK,TS val 729869651 ecr 0,nop,wscale 8], length 0
17:54:14.954763 IP 198.199.98.246.40125 > 10.0.0.10.32400: Flags [S], seq 1528187397, win 14600, options [mss 1460,sackOK,TS val 729869901 ecr 0,nop,wscale 8], length 0
17:54:14.989988 IP 198.199.98.246.40128 > 10.0.0.10.32400: Flags [S], seq 1176045227, win 14600, options [mss 1460,sackOK,TS val 729869909 ecr 0,nop,wscale 8], length 0
17:54:15.986789 IP 198.199.98.246.40128 > 10.0.0.10.32400: Flags [S], seq 1176045227, win 14600, options [mss 1460,sackOK,TS val 729870159 ecr 0,nop,wscale 8], length 0


Now that we know the IP address of the host coming thru the VPS we see the final host (destination NAT 172.20.0.20 responding to the public IP address):

Code Select Expand

# tcpdump -i vtnet0_vlan700 host 198.199.98.246 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0_vlan700, link-type EN10MB (Ethernet), capture size 262144 bytes
12:58:26.032396 IP 198.199.98.246.41128 > 172.20.0.20.32400: Flags [S], seq 1186752319, win 14600, options [mss 1460,sackOK,TS val 729932656 ecr 0,nop,wscale 8], length 0
12:58:26.032596 IP 172.20.0.20.32400 > 198.199.98.246.41128: Flags [S.], seq 3453310492, ack 1186752320, win 65160, options [mss 1460,sackOK,TS val 3269064663 ecr 729932656,nop,wscale 7], length 0
12:58:27.030216 IP 198.199.98.246.41128 > 172.20.0.20.32400: Flags [S], seq 1186752319, win 14600, options [mss 1460,sackOK,TS val 729932906 ecr 0,nop,wscale 8], length 0
12:58:27.030379 IP 172.20.0.20.32400 > 198.199.98.246.41128: Flags [S.], seq 3453310492, ack 1186752320, win 65160, options [mss 1460,sackOK,TS val 3269065661 ecr 729932656,nop,wscale 7], length 0
12:58:27.078113 IP 198.199.98.246.41131 > 172.20.0.20.32400: Flags [S], seq 1180124324, win 14600, options [mss 1460,sackOK,TS val 729932918 ecr 0,nop,wscale 8], length 0
12:58:27.078249 IP 172.20.0.20.32400 > 198.199.98.246.41131: Flags [S.], seq 1408018903, ack 1180124325, win 65160, options [mss 1460,sackOK,TS val 3269065708 ecr 729932918,nop,wscale 7], length 0
12:58:28.044387 IP 172.20.0.20.32400 > 198.199.98.246.41128: Flags [S.], seq 3453310492, ack 1186752320, win 65160, options [mss 1460,sackOK,TS val 3269066675 ecr 729932656,nop,wscale 7], length 0
12:58:28.074094 IP 198.199.98.246.41131 > 172.20.0.20.32400: Flags [S], seq 1180124324, win 14600, options [mss 1460,sackOK,TS val 729933168 ecr 0,nop,wscale 8], length 0
12:58:28.074298 IP 172.20.0.20.32400 > 198.199.98.246.41131: Flags [S.], seq 1408018903, ack 1180124325, win 65160, options [mss 1460,sackOK,TS val 3269066704 ecr 729932918,nop,wscale 7], length 0
12:58:28.124204 IP 198.199.98.246.41134 > 172.20.0.20.32400: Flags [S], seq 260678019, win 14600, options [mss 1460,sackOK,TS val 729933180 ecr 0,nop,wscale 8], length 0
12:58:28.124316 IP 172.20.0.20.32400 > 198.199.98.246.41134: Flags [S.], seq 2906162909, ack 260678020, win 65160, options [mss 1460,sackOK,TS val 3269066755 ecr 729933180,nop,wscale 7], length 0
12:58:29.100367 IP 172.20.0.20.32400 > 198.199.98.246.41131: Flags [S.], seq 1408018903, ack 1180124325, win 65160, options [mss 1460,sackOK,TS val 3269067731 ecr 729932918,nop,wscale 7], length 0
12:58:29.120326 IP 198.199.98.246.41134 > 172.20.0.20.32400: Flags [S], seq 260678019, win 14600, options [mss 1460,sackOK,TS val 729933430 ecr 0,nop,wscale 8], length 0
12:58:29.120921 IP 172.20.0.20.32400 > 198.199.98.246.41134: Flags [S.], seq 2906162909, ack 260678020, win 65160, options [mss 1460,sackOK,TS val 3269067751 ecr 729933180,nop,wscale 7], length 0
12:58:30.064379 IP 172.20.0.20.32400 > 198.199.98.246.41128: Flags [S.], seq 3453310492, ack 1186752320, win 65160, options [mss 1460,sackOK,TS val 3269068695 ecr 729932656,nop,wscale 7], length 0
12:58:30.124414 IP 172.20.0.20.32400 > 198.199.98.246.41134: Flags [S.], seq 2906162909, ack 260678020, win 65160, options [mss 1460,sackOK,TS val 3269068755 ecr 729933180,nop,wscale 7], length 0
12:58:31.116373 IP 172.20.0.20.32400 > 198.199.98.246.41131: Flags [S.], seq 1408018903, ack 1180124325, win 65160, options [mss 1460,sackOK,TS val 3269069747 ecr 729932918,nop,wscale 7], length 0
12:58:32.140411 IP 172.20.0.20.32400 > 198.199.98.246.41134: Flags [S.], seq 2906162909, ack 260678020, win 65160, options [mss 1460,sackOK,TS val 3269070771 ecr 729933180,nop,wscale 7], length 0



Problem is that wg0 is where opnsense should be sending those responses and it should outbound NAT 172.20.0.20 as 10.0.0.10. Routing this via wg0 seems to be not happening and instead going thru opnsense-WAN.

https://imgur.com/a/vZQyGvE << configs

Here is confirmation the responses are going thru default route / opnsense WAN and screenshots of the policy based rules that should have forced the traffic thru wg0 wireguard interface.

Code Select Expand

# tcpdump -i em0 host 198.199.98.246 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:02:11.952841 IP 10.0.0.10.32400 > 198.199.98.246.42000: Flags [S.], seq 1206282859, ack 3275283905, win 65160, options [mss 1460,sackOK,TS val 3269290582 ecr 729989138,nop,wscale 7], length 0
13:02:12.958465 IP 10.0.0.10.32400 > 198.199.98.246.42000: Flags [S.], seq 1206282859, ack 3275283905, win 65160, options [mss 1460,sackOK,TS val 3269291588 ecr 729989138,nop,wscale 7], length 0
13:02:12.999633 IP 10.0.0.10.32400 > 198.199.98.246.42004: Flags [S.], seq 965419102, ack 1952030132, win 65160, options [mss 1460,sackOK,TS val 3269291629 ecr 729989400,nop,wscale 7], length 0
13:02:13.964997 IP 10.0.0.10.32400 > 198.199.98.246.42000: Flags [S.], seq 1206282859, ack 3275283905, win 65160, options [mss 1460,sackOK,TS val 3269292595 ecr 729989138,nop,wscale 7], length 0
13:02:13.999727 IP 10.0.0.10.32400 > 198.199.98.246.42004: Flags [S.], seq 965419102, ack 1952030132, win 65160, options [mss 1460,sackOK,TS val 3269292629 ecr 729989400,nop,wscale 7], length 0
13:02:14.041662 IP 10.0.0.10.32400 > 198.199.98.246.42010: Flags [S.], seq 2726232845, ack 434760706, win 65160, options [mss 1460,sackOK,TS val 3269292671 ecr 729989661,nop,wscale 7], length 0
13:02:15.021120 IP 10.0.0.10.32400 > 198.199.98.246.42004: Flags [S.], seq 965419102, ack 1952030132, win 65160, options [mss 1460,sackOK,TS val 3269293651 ecr 729989400,nop,wscale 7], length 0
13:02:15.045706 IP 10.0.0.10.32400 > 198.199.98.246.42010: Flags [S.], seq 2726232845, ack 434760706, win 65160, options [mss 1460,sackOK,TS val 3269293675 ecr 729989661,nop,wscale 7], length 0
13:02:15.981011 IP 10.0.0.10.32400 > 198.199.98.246.42000: Flags [S.], seq 1206282859, ack 3275283905, win 65160, options [mss 1460,sackOK,TS val 3269294611 ecr 729989138,nop,wscale 7], length 0
13:02:16.076957 IP 10.0.0.10.32400 > 198.199.98.246.42010: Flags [S.], seq 2726232845, ack 434760706, win 65160, options [mss 1460,sackOK,TS val 3269294707 ecr 729989661,nop,wscale 7], length 0
13:02:17.036976 IP 10.0.0.10.32400 > 198.199.98.246.42004: Flags [S.], seq 965419102, ack 1952030132, win 65160, options [mss 1460,sackOK,TS val 3269295667 ecr 729989400,nop,wscale 7], length 0
13:02:18.093017 IP 10.0.0.10.32400 > 198.199.98.246.42010: Flags [S.], seq 2726232845, ack 434760706, win 65160, options [mss 1460,sackOK,TS val 3269296723 ecr 729989661,nop,wscale 7], length 0


Hi,

I'm wondering if the 21.1 version may have the kernel module for wireguard rather than the golang version?

It seems like FreeBSD 13 will have it soon if https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-WireGuard-Lands is accurate.

I miss the ability to do CTRL + R and regex search past commands in linux terminal.

It seems like opnsense implements shell at /usr/local/sbin/opnsense-shell << is it possible to add the ability to CTRL+R regex search in this shell?

Item 2 in this guide shows a GIF of what I mean by reverse-i-search: https://thomasweng.com/terminal_tips/

Thanks

I'm trying to give public internet routable IPv6 address space to clients in one of my opnsense interfaces "V6only"

This interface has been configured with IPv4 disabled and V6 only. My WAN interface obtains V6 addresses from ISP. My goal is to have a network where clients get an IPv6 address that is routable (pingable) from the internet to my network.

I have been struggling to get my windows 10 VM to get an IPv6 from the same address block than my ISP gives my WAN - can I get some help? Not sure what I am doing wrong.

Here's pictures of my interface settings, firewall rules (allow any any ipv6) and interface overviews. https://imgur.com/a/tMrWwTN

I have two opnsense installs:
- True ISP WAN let's call it "primary"
- and "backup" opnsense which has a bunch of VMs in it.

"backup" WAN is connected to "primary" LAN network.
On Windows I noticed the "backup" is broadcasting some sensitive information to the outside world about itself: serial number, unique identifier, software version as "FreeBSD router"

How do I stop this and where is it configured?

I have "UPnP and NAT-PMP" working just fine on my regular WAN when traffic is from LAN.

I have a new interface for my VPN_WAN which I am successfully using as the gateway for some hosts on my network. I am wondering how to make sure that UPNP can work as it does on my LAN but with this new gateway?

Are there any special settings? I am not sure about adding my VPN_WAN to my "External interfaces" which already contains "WAN" - what is the right path?

I bought an Intel Pro Dual gigabit NIC that is not being detected by opnsense. Installing FreeBSD 12 and it works out of the box without needing a driver.

I tried to compile the if_em.ko from source then copy to /boot/kernerl but kldload complained.

I found an older discussion for 16.7 where some steps were shared - I thought the "os-intel" package was created for this driver? I don't see it anywhere now.

https://forum.opnsense.org/index.php?topic=3706.0
https://forum.opnsense.org/index.php?topic=3689.0

Here is the NIC being detected just fine in BSD 12:

---<<BOOT>>---
Copyright (c) 1992-2020 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.2-STABLE r368787 GENERIC amd64
FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
VT(vga): text 80x25
CPU: Common KVM processor (2200.05-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0xf61  Family=0xf  Model=0x6  Stepping=1
  Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x80202001<SSE3,CX16,x2APIC,HV>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
Hypervisor: Origin = "KVMKVMKVM"
real memory  = 8589934592 (8192 MB)
avail memory = 8244654080 (7862 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <BOCHS  BXPCAPIC>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-23 on motherboard
Launching APs: 2 1 3
random: entropy device external interface
kbd1 at kbdmux0
000.000023 [4336] netmap_init               netmap: loaded module
[ath_hal] loaded
module_register_init: MOD_LOAD (vesa, 0xffffffff811187c0, 0) error 19
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <BOCHS BXPCRSDT> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> mem 0xfb000000-0xfbffffff,0xfea14000-0xfea14fff at device 1.0 on pci0
vgapci0: Boot video device
uhci0: <Intel 82801I (ICH9) USB controller> port 0x6040-0x605f irq 16 at device 26.0 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
uhci1: <Intel 82801I (ICH9) USB controller> port 0x6060-0x607f irq 17 at device 26.1 on pci0
usbus1 on uhci1
usbus1: 12Mbps Full Speed USB v1.0
uhci2: <Intel 82801I (ICH9) USB controller> port 0x6080-0x609f irq 18 at device 26.2 on pci0
usbus2 on uhci2
usbus2: 12Mbps Full Speed USB v1.0
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xfea15000-0xfea15fff irq 19 at device 26.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
usbus3: 480Mbps High Speed USB v2.0
hdac0: <Intel 82801I HDA Controller> mem 0xfea10000-0xfea13fff irq 16 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> mem 0xfea16000-0xfea16fff irq 16 at device 28.0 on pci0
pcib1: [GIANT-LOCKED]
pci1: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection> port 0x5000-0x501f mem 0xfe800000-0xfe81ffff,0xfe820000-0xfe83ffff irq 16 at device 0.0 on pci1
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Using an MSI interrupt
em0: Ethernet address: 00:26:55:d9:34:7e
em0: netmap queues/slots: TX 1/1024, RX 1/1024
em1: <Intel(R) PRO/1000 Network Connection> port 0x5020-0x503f mem 0xfe860000-0xfe87ffff,0xfe880000-0xfe89ffff irq 17 at device 0.1 on pci1
em1: Using 1024 TX descriptors and 1024 RX descriptors
em1: Using an MSI interrupt
em1: Ethernet address: 00:26:55:d9:34:7f
em1: netmap queues/slots: TX 1/1024, RX 1/1024
pcib2: <PCI-PCI bridge> mem 0xfea17000-0xfea17fff irq 16 at device 28.1 on pci0
pcib2: [GIANT-LOCKED]
pcib3: <PCI-PCI bridge> mem 0xfea18000-0xfea18fff irq 16 at device 28.2 on pci0
pcib3: [GIANT-LOCKED]
pcib4: <PCI-PCI bridge> mem 0xfea19000-0xfea19fff irq 16 at device 28.3 on pci0
pcib4: [GIANT-LOCKED]
uhci3: <Intel 82801I (ICH9) USB controller> port 0x60a0-0x60bf irq 16 at device 29.0 on pci0
usbus4 on uhci3
usbus4: 12Mbps Full Speed USB v1.0
uhci4: <Intel 82801I (ICH9) USB controller> port 0x60c0-0x60df irq 17 at device 29.1 on pci0
usbus5 on uhci4
usbus5: 12Mbps Full Speed USB v1.0
uhci5: <Intel 82801I (ICH9) USB controller> port 0x60e0-0x60ff irq 18 at device 29.2 on pci0
usbus6 on uhci5
usbus6: 12Mbps Full Speed USB v1.0
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xfea1a000-0xfea1afff irq 19 at device 29.7 on pci0
usbus7: EHCI version 1.0
usbus7 on ehci1
usbus7: 480Mbps High Speed USB v2.0
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci2: <ACPI PCI bus> on pcib5
pcib6: <PCI-PCI bridge> mem 0xfe000000-0xfe0000ff irq 21 at device 1.0 on pci2
pci3: <PCI bus> on pcib6
virtio_pci0: <VirtIO PCI Balloon adapter> port 0x4000-0x403f mem 0xfc600000-0xfc603fff irq 20 at device 3.0 on pci3
vtballoon0: <VirtIO Balloon Adapter> on virtio_pci0
virtio_pci1: <VirtIO PCI SCSI adapter> port 0x4040-0x407f mem 0xfde40000-0xfde40fff,0xfc604000-0xfc607fff irq 22 at device 5.0 on pci3
vtscsi0: <VirtIO SCSI Adapter> on virtio_pci1
virtio_pci2: <VirtIO PCI Network adapter> port 0x4080-0x409f mem 0xfde41000-0xfde41fff,0xfc608000-0xfc60bfff irq 23 at device 18.0 on pci3
vtnet0: <VirtIO Networking Adapter> on virtio_pci2
vtnet0: Ethernet address: 6a:41:56:fa:65:69
vtnet0: netmap queues/slots: TX 1/256, RX 1/128
000.000114 [ 445] vtnet_netmap_attach       vtnet attached txq=1, txd=256 rxq=1, rxd=128
pcib7: <PCI-PCI bridge> mem 0xfe001000-0xfe0010ff irq 22 at device 2.0 on pci2
pci4: <PCI bus> on pcib7
pcib8: <PCI-PCI bridge> mem 0xfe002000-0xfe0020ff irq 23 at device 3.0 on pci2
pci5: <PCI bus> on pcib8
pcib9: <PCI-PCI bridge> mem 0xfe003000-0xfe0030ff irq 20 at device 4.0 on pci2
pci6: <PCI bus> on pcib9
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel ICH9 AHCI SATA controller> port 0x6100-0x611f mem 0xfea1b000-0xfea1bfff irq 16 at device 31.2 on pci0
ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> port 0xcd8-0xce3 on acpi0
acpi_syscontainer2: <System Container> port 0x620-0x62f on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
orm0: <ISA Option ROMs> at iomem 0xca000-0xcafff,0xcb000-0xcbfff,0xea000-0xeffff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
attimer0: non-PNP ISA device will be removed from GENERIC in FreeBSD 12.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
Trying to mount root from zfs:zroot/ROOT/default []...
Root mount waiting for: usbus0 usbus1 usbus2 usbus3 usbus4 usbus5 usbus6 usbus7 CAM
ugen2.1: <Intel UHCI root HUB> at usbus2
ugen7.1: <Intel EHCI root HUB> at usbus7
ugen6.1: <Intel UHCI root HUB> at usbus6
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
ugen3.1: <Intel EHCI root HUB> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen0.1: <Intel UHCI root HUB> at usbus0
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen5.1: <Intel UHCI root HUB> at usbus5
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen1.1: <Intel UHCI root HUB> at usbus1
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen4.1: <Intel UHCI root HUB> at usbus4
uhub7: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
da0 at vtscsi0 bus 0 scbus0 target 0 lun 0
da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 32768MB (67108864 512 byte sectors)
cd0 at ahcich1 bus 0 scbus2 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: 0MB (1 0 byte sectors)
uhub2: 2 ports with 2 removable, self powered
uhub7: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
Root mount waiting for: usbus3 usbus7
Root mount waiting for: usbus3 usbus7
uhub3: 6 ports with 6 removable, self powered
uhub1: 6 ports with 6 removable, self powered
ugen7.2: <QEMU QEMU USB Tablet> at usbus7
lo0: link state changed to UP
vtnet0: link state changed to UP
uhid0 on uhub1
uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 2> on usbus7
root@freebsd12:/home/gfm #

Hi!

New to opnsense and I am trying to figure out why under "Live View" under Firewall I am not seeing any block events.

I believe the firewall by default drops all packets without an explicit accept, is this why?

I am trying to troubleshoot a new host which sits on a VLAN under the firewall. I can see the traffic exiting the DMZ vlan but don't see where the block happens.

I am interested in seeing 'block' events because it helps me craft a specific firewall rule to allow the traffic.

is there a better way? is it possible to enable a more verbose 'debug' mode temporarily?

I am curious if this [1] documentation is still accurate. Transparent Bridge + Traffic shaping is not possible in 18.1?

[1] https://docs.opnsense.org/manual/how-tos/transparent_bridge.html

Howdy,

I have been a pfsense user for 3 years (and have the pfsense book), I recently became aware of opnsense and noticed there's been a lot of effort put into this fork of pfsense, I wanted to come here and ask to see if I get some direction and help on my upcoming project where I will need an open source solution.

The book doesn't really talk about how to make your pfsense firewall be transparent and allow public IP vlans - there is some documentation (a PDF from someone 4 years ago here https://forum.pfsense.org/index.php?topic=50711.0)

I don't know if anyone, maybe the developers of opnsense have a similar setup to what I will need to setup (see attached diagram)

There is a layer 2 dumb switch that can segregate ports by 802.1q tags, but basically the goal of what I would like to do by possibly running opnsense is for it to allow a seamless/transparent firewall mode, with the ability to allow all network ports and services (at first) and later allowing me to lock down ports/services on a per VLAN basis.

I think this could be easily done with 'rules' - I am guessing the default rule is to block-all traffic unless explicitly allowed? I am not sure how to make the firewall transparent if I have a rule allow any to all for that vlan and later add "deny" rules on top...

Would OPNsense be a better option for me? is there a better or has someone done this with OPNsense before? Basically it will be the firewall protecting a small datacenter.