Messages

So looking again at the NAT settings, seems like there are 3 options in the drop down for `ixl0`:

* interface address
* ONTWAN address -- ONTWAN is the name for ixl0
* 192.168.11.10 ONT Management -- the virtual IP

Seems like using the 2nd option should provide the correct results?   Or are the 1st and 2nd effectively the same option with different names?

Meant to mention that for outbound NAT I'm using "Manual outbound NAT rule generation" and selecting NAT Address: "interface address"

This was happening on 26.1.x and continues on 26.7.1.   I'm on AT&T fiber and using an AZORES WAS-110 fiber module/ONT to bypass the AT&T provided gateway.  For this to work, my `ixl0` interface uses DHCP/DHCPv6 to get IP addresses and I create a virtual IP (192.168.11.10/24) on `ixl0` in order to manage the WAS-110 module.

The problem is that for whatever reason, getting an IP address from AT&T is _slow_.  After rebooting the firewall, I often have to login and manually run `dhclient ixl0` to force it to try again and that works.  Once an IPv4 address is obtained, DHCPv6 seems to happen automatically.

The problem is that the system seems to not really register that an IPv4 address has been obtained and so NAT is broken- running `tcpdump -ni ixl0` shows traffic egressing with a source IP of 192.168.11.10- not the public IP that AT&T has given via DHCP.  I've figured out that I can fix this by going into the interface config setting and re-applying the existing settings, but this sucks.

Seems like the root cause of the issue is AT&T being slow to give out an IP address over DHCP- any way to make things more robust?

Oh, this seems to be an issue with running my collector under docker with a docker network.  TL;DR: NAT sucks.

Running 25.7.4.

Enabled netflow collection on 2 physical interfaces + 4 VLAN tagged interfaces on one of the 2 physical interfaces.  Disabled capture local.

Sending the netflow data to a collector running on another box, and seeing far too many individual exporters coming from my opnsense box.  Seems like the UDP source port keeps rotating which means my collector is constantly dropping data as it is waiting for a new template to be sent.  Often, the port rotates again even before a template is sent?

This sample is from only a few minutes.  I have seen over 600 unique source ports after letting it run over night.

welp was definitely software related.  ended up re-installing from scratch and problem resolved itself.  dunno how I could of broken things.

installed iperf3 on the firewall and my NAS (both are 10Gbps):

iperf3 -p 50419 -c thewall.xxxxxxx
Connecting to host thewall.xxxxxxx, port 50419
[  5] local 172.16.1.90 port 51190 connected to 172.16.1.1 port 50419
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   414 MBytes  3.48 Gbits/sec    0    128 KBytes
[  5]   1.00-2.00   sec   428 MBytes  3.59 Gbits/sec    0    128 KBytes
[  5]   2.00-3.00   sec   369 MBytes  3.10 Gbits/sec    0    128 KBytes
[  5]   3.00-4.00   sec   446 MBytes  3.75 Gbits/sec    0    128 KBytes
[  5]   4.00-5.00   sec   402 MBytes  3.37 Gbits/sec    0    128 KBytes
[  5]   5.00-6.00   sec   387 MBytes  3.24 Gbits/sec    0    128 KBytes
[  5]   6.00-7.00   sec   441 MBytes  3.70 Gbits/sec    0    128 KBytes
[  5]   7.00-8.00   sec   417 MBytes  3.49 Gbits/sec    0    128 KBytes
[  5]   8.00-9.00   sec   413 MBytes  3.47 Gbits/sec    0    128 KBytes
[  5]   9.00-10.00  sec   408 MBytes  3.42 Gbits/sec    0    128 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.03 GBytes  3.46 Gbits/sec    0             sender
[  5]   0.00-10.53  sec  4.03 GBytes  3.29 Gbits/sec                  receiver

Both hosts are 1500 byte MTU.  So it's not a NIC/driver issue right?  I don't have IDS or any heavy CPU load running on the firewall and when I run speedtest from the NAS the CPU load on the firewall is ~0.4.

speedtest-go running on the firewall:

./speedtest-go

    speedtest-go v1.7.10 (git-1395781) @showwin

✓ ISP: xx.x.x.x (AT&T Internet) [xxxxx, xxxx]
✓ Found 21 Public Servers

✓ Test Server: [17846] 6.30km San Jose, CA (United States) by Sonic.net, Inc.
✓ Latency: 6.949538ms Jitter: 742.116µs Min: 5.804711ms Max: 8.060521ms
✓ Packet Loss Analyzer: Running in background (<= 30 Secs)
✓ Download: 2047.58 Mbps (Used: 2465.56MB) (Latency: 12ms Jitter: 6ms Min: 6ms Max: 24ms)
✓ Upload: 2008.28 Mbps (Used: 2457.25MB) (Latency: 13ms Jitter: 4ms Min: 5ms Max: 22ms)
✓ Packet Loss: 0.00% (Sent: 283/Dup: 0/Max: 282)

speedtest-go running on the NAS (note the performance is higher than 400Mbps... seems like the go implimentation is faster than Python? Not seeing this good normally, but it's still < 1Gbps):

./speedtest-go

    speedtest-go v1.7.10 (git-1395781) @showwin

✓ ISP: xx.x.x.x (AT&T Internet) [xxxx, xxxx]
✓ Found 21 Public Servers

✓ Test Server: [56175] 6.30km San Jose, CA (United States) by Acreto
✓ Latency: 5.164619ms Jitter: 773.843µs Min: 4.187189ms Max: 6.889106ms
✓ Packet Loss Analyzer: Running in background (<= 30 Secs)
✓ Download: 742.47 Mbps (Used: 929.82MB) (Latency: 8ms Jitter: 3ms Min: 4ms Max: 15ms)
✓ Upload: 833.61 Mbps (Used: 1072.77MB) (Latency: 10ms Jitter: 4ms Min: 4ms Max: 18ms)
✓ Packet Loss: 0.00% (Sent: 268/Dup: 0/Max: 267)

Recently upgraded my hardware from a Intel i3 7100U running pfSense to protectli VP6630 (i3-1215U) running latest OPNsense.  Internet provided by AT&T fiber (2Gbps service). 

TL;DR: my old firewall has no problems forwarding at line rate 1Gbps (it only has 1GbE NICs).  But the new firewall with 2.5GbE NIC (Intel i226-V) uplink to AT&T and 10Gbps SFP+ connection to my switch is forwarding at ~400Mbps.  I've manually installed speedtest client on the OPNsense box and it's showing >2Gbps so I know it's not some negotiation issue.   I've also tried doing a 3x1Gbps LACP/layer4 on the LAN side (how my old pfSense box was configured) and that had the same problem.  I know there's been some talk of issues with the i225-V/i226-V that sound similar, but the fact that it can send/receive at basically line rate seems to indicate it's a forwarding issue?

Firewall rules are fairly simple.  No rate limiting policies or anything like that.  I've tried enabling/disabling all the hardware offloading features under Interfaces -> Settings.  Not sure what to try next?