Messages

[...]Do I need a jumper or something?[...]

Given that the quote is "PCIe x4", I assume both sockets are M-keyed... There are shared signals between PCI-e and SATA, but it may auto-detect. I may have even tested it unknowingly - I'd have to go look at my pile o'motherboards. They're signal anyway - it should not be possible to damage an M-keyed SSD by sticking it into an M or M+B slot. I'd definitely look at the link width and version via "pciconf -lbcevV [device]" (format could be wrong) (assuming it's detected).

I'd punish you by sending you all of my old Aquantia and Tehuti hardware, but the joke wouldn't be worth the ridiculous international shipping.

It might not be worth it, but you should be able to set the PCI-e version in the BIOS. (I can find it easily, but only if I'm looking at it.)

Have default sysctls/tuneables changed? Check (shell: "sysctl net.inet.tcp" or "sysctl [individual setting]"):

net.inet.tcp.mssdflt: 1460
net.inet.tcp.v6mssdflt: 1440
net.inet.tcp.minmss: 536
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.pmtud_blackhole_detection: 0
net.inet.tcp.pmtud_blackhole_mss: 1460
net.inet.tcp.v6pmtud_blackhole_mss: 1440

Those are... not my current values, but what I expect to set when I change firewalls. Not sure if I'll enable blackhole detection - need to read up on it. (Configuring the blackhole MSS settings explicitly is generally not necessary.) Of course these normally only affect TCP terminating on the firewall - I would not expect them to affect sessions initiated from clients behind the firewall. Clients would be affected by an MSS clamp but those generally use explicit values (in the interface settings) and are not derived from system MSS settings. Set them on OPNsense under "System: Settings: Tunables". (The v6 settings may be configured from the v4 settings by default, but I haven't checked that.)

I don't think these are your actual issue, but it can't hurt to check 'em.

Also, the wireless radio support. There are a few, but very few for later 802.11 revisions. Then you can get into wacky compatibility issues with Intel radios. Not saying it's impossible to set up, but dedicated hardware is generally easier.

I don't see a mention of NAT or a route for the LAN network on the PPPoE (upstream) provider device. One or the other would normally be required. (I also didn't see the PPPoE-assigned IP offhand.)

Have a look at Hostwatch - high disk writes.

I don't know if the locations are the same, but this is usually due to using a RAM disk for logging under "System: Settings: Miscellaneous" -> "Disk / Memory Settings (reboot to apply changes)". Not the root cause, but most systems have more storage than RAM, and ZFS compresses logs very effectively. You can then hunt down the root cause without worrying (so much) about crashing.

Have a look at Investigating outbound rules in OPNsense. As for order, I haven't gone to v26 yet, but there are ordering options available that may be usable for you (Let's talk firewall rule order ...).

[...]This is what you are advising me to check, correct?

Essentially. Not the capture (at this point), but what was learned on each device. Normally proxies are pretty easy to spot... once you look. More of an issue on bridges because of the (potentially) larger L2 domain.

Have you checked the MAC addresses learned from ARP on each device? Actual values, not just presence. Looking for a problem proxy.

I may not be following your precise configuration. Is the AP in bridge mode?

Context: I have a similar setup: multiple bridges on an OPNsense firewall, with a Linksys MX8500 running OpenWRT connected to one bridge. The MX8500 is also in bridge mode - that is, the wireless radios are attached to a bridge which contains the Ethernet interface connected to the firewall; it (the radio bridge on the AP) has no IP address assigned and no DHCP client (or server, of course) running. (For completeness I have a second bridge on the MX8500 for management which is not accessible from the client/radio bridge, which is a DHCP client, connected to a different bridge on the firewall.)

I hadn't thought of differences in utilities - how does it look in top, for instance?

Code Select Expand

[...]
Mem: 101M Active, 363M Inact, 2732M Wired, 104K Buf, 27G Free
ARC: 1357M Total, 767M MFU, 407M MRU, 9081K Anon, 23M Header, 149M Other
    1047M Compressed, 3182M Uncompressed, 3.04:1 Ratio
[...]

The ARC allocation (generally) grows from boot to a stable level. Some memory utilization reports neglect/subtract ARC (e.g. the percentage reported in the dashboard widget), so memory usage may appear more stable. So yes, a characteristic of FreeBSD (with ZFS, at least).

That is an 10G Base-SR type SFP+ transceiver, which is way less power-consuming than RJ45 ones. For DAC cables, you do not even get a temp reading, because they are the least power hungry of the three types.

I was just demonstrating the command to read the temp, if supported. Slipped my mind earlier. Heh - as for DAC cables, the only active component they have is the ID chippie. It might be interesting to read the ambient temperature of the cage, but I don't know that I'd want to pay extra for it.

[...]SFPs have often a MON[...]

Grr. Good point:

Code Select Expand

root@fw:/home/user # ifconfig -v ixl3
ixl3: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: x710p3 (opt4)
        options=4800028<VLAN_MTU,JUMBO_MTU,HWSTATS,MEXTPG>
        ether 3c:fd:fe:e7:2d:8b
        media: Ethernet autoselect (10Gbase-SR <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>
        drivername: ixl3
        plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
        vendor: Intel Corp PN: AFBR-709DMZ-IN3 SN: AA202830LM3 DATE: 2020-07-11
        module temperature: 27.90 C voltage: 3.35 Volts
        lane 1: RX power: 0.51 mW (-2.92 dBm) TX bias: 5.46 mA
root@fw:/home/user #

[...]If appliance is powered down for 5 minutes or so, full PPPOE speeds return[...]

That does sound like it's cooling off, but if that's the case, it should be consistent. Do you have any other connectivity options? e.g. a DAC cable and a switch with 10GBASE-T and SFP+ ports.