Messages

Deciso could improve their datasheets by mentioning the used chipsets for the ethernet ports. I guess the 2.5G ports use Intel I226-V or I226-LM controllers?

Both the XG-7100 1U and DT seem to have an Intel Atom C3558 CPU.

The controller is X553 from the X550 Series, driver ix. Seeems Netgate created a custom configuration for their device.

There is a Marvell 88E6190 10(11)-Port Switch connected to 2x2.5G internal X553 ports.

https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/switch-overview.html

So you would probably only see ix0-ix3 in OPNsense.

Hardware support for the XG-7100, including:
C3000 NIC support (factory installations only)
C3000 SoC support (factory installations only)
Marvell 88E6190 switch support (factory installations only)

https://github.com/pfsense/docs/blob/74ca3ce0d8844cb925399ccbe674504c6562fb1d/source/releases/2-4-3-new-features-and-changes.rst#L47

Integrated Switch Improvements: Netgate devices with integrated switches such as the SG-3100 and XG-7100 can now configure per-port speed and duplex settings, discrete port configuration interfaces can now be tied to switch ports for up/down status, and LAGG support is also now available (Load Balance mode only)

https://github.com/pfsense/docs/blob/74ca3ce0d8844cb925399ccbe674504c6562fb1d/source/releases/2-4-4-new-features-and-changes.rst#L51

The Marvell 88E6190 is supported according to the OPNsense github repository:

https://github.com/opnsense/src/blob/08aa869336f6a7c19d5bbcaa61269a3f21ca1993/sys/dev/etherswitch/e6000sw/e6000sw.c#L273

Maybe just try it.

A related thread:

https://forum.opnsense.org/index.php?topic=32991.0

First of all, thank you vpx23!

You're welcome!

Unfortunately the manual of the ASRock Rack B650D4U-2L2T/BCM does not speak of bifurcation, but there is an option inside the AMD PBS submenu called PCIE/GFX Lanes Configuration that can be set to x8x8.

That sounds like the right thing to me but just ask ASRock to be 100% sure before buying the E810-2CQDA2.

Du kannst mal versuchen den lighttpd Webserver in der Konsole neuzustarten wie hier beschrieben:

Code Select Expand

/usr/local/etc/rc.restart_webgui

oder im Konsolen-Menü:

Code Select Expand

  11)     Restart web interface

https://forum.opnsense.org/index.php?topic=1188.0

I guess you don't have an ideal setup to do these tests.

iperf3 client<--->test object<--->iperf3 server

And ideally both client and server are very powerful and only run dedicated iperf3, so no other process will disturb the throughput. The test object should always be the bottleneck.

When you have 3 systems with weak CPUs in the setup you will have 3 potential bottlenecks. Even worse when the test object itself is the iperf3 client or server.

OK, now I saw the extra board, it's probably only for power because I don't see any data cable.

This must be the PCIe to PCI Bridge Controller on your mainboard:

https://www.asmedia.com.tw/product/E8AyQ44sX3zPbruF/2f2yq89xz7URBZqD

It converts PCIe 1.1 x1 (0.250 GByte/s) to 2x PCI 32-bit with 33 MHz. I don't know why they made it so slow because the board is from 2012 when PCI 3.0 was already 8 years old.

According to your output the Intel PRO/1000 MT 82546EB are PCI-X 1.0 | 64 Bit | 133 MHz | 1.066 GByte/s | 3.3V. So they are not the problem, they should easily reach 940 MBit/s. Even with both ports and full-duplex at the same time (3,760 MBit/s) they wouldn't saturate the possible bandwidth of PCI-X. Only the 4 port cards with 7,520 Mbit/s + overhead could saturate the max. 8,528 MBit/s bandwidth.

That's a crazy setup, 2 PCI-X dual port cards in one PCI slot.

But the PCI slot on the board is definitely 32 bit because this is shorter than a 64 bit port, so the max. speed would be 0.266 GByte/s.

Did you try the speed tests with only one NIC plugged?

To find out the PCI port speed we could try these commands in OPNsense:

Code Select Expand

pciconf -lvc

Code Select Expand

lspci -vvv

https://unix.stackexchange.com/questions/541688/how-to-see-pcie-device-info-on-freebsd

What do you mean with paddleboard, could you show a photo please?

If you're in Austria, Germany, Swiss or Belgium you can already buy it, I also couldn't find it anywhere else.

I226T1BLK
GTIN/EAN: 5032037262668
https://www.heise.de/preisvergleich/4394281832
https://shop.scheuss-partner.ch/de/products/5114445-i226t1blk-intel-ethernet-network-adapter-i226-t1
https://www.lasystems.be/fr/intel-i226t1blk

I226T1
GTIN/EAN: 5032037250399
https://www.heise.de/preisvergleich/4394280207
https://shop.scheuss-partner.ch/de/products/5114443-i226t1-intel-ethernet-network-adapter-i226-t1
https://www.lasystems.be/fr/intel-i226t1

I looked at photos of the mainboard and the PCI slots seem to be PCI 2.3 | 32 Bit | 33 MHz | 0.133 GByte/s | 5V while the NICs probably are PCI-X 1.0 | 64 Bit | 66 MHz | 0.533 GByte/s | 3.3V.

PCI-X is backward compatible, that's why it works, but there must be some overhang from the PCI-X cards at the end of the slot, could you please make a photo?

So the theoretical bandwidth (unidirectional) = 0.133 GByte/s * 8 = 1.064 Gbit/s shared by 2 ports = 0.532 GBit/s.

A PCI Express to PCI Extended adapter would be nice but this only seems to exist as PCIe to PCI 32 Bit 5 V adapter.

There's a brand new Intel I226-T1 NIC with the I226-LM controller, maybe they fixed the issues in the latest revision. But the issues I read were only in relation to consumer mainboards with Intel 700 Series chipsets.

https://www.intel.com/content/www/us/en/products/sku/225771/intel-ethernet-network-adapter-i226t1/specifications.html

The only other 2.5 GBit/s NICs I know are with Aquantia (now Marvell) chipsets.

https://www.marvell.com/products/ethernet-adapters-and-controllers/fastlinq-edge-ethernet-controllers.html

Broadcom doesn't seem to have 2.5 GBit/s NICs because they propbably aim more towards business users.

I already recommended another NIC in the German thread but before you buy a new one did you try the Realtek driver instead of the FreeBSD driver?

It's "os-realtek-re" under System: Firmware: Plugins.

Schau dir mal den kleinen Chip mit dem Realtek-Logo auf der Netzwerkkarte an, da steht sicherlich RTL8169E drauf?

Dafür sprechen auch die letzten Zeilen in diesem Output:

https://linuxreviews.org/TP-LINK_TG-3468_PCIe_Gigabit_Adapter

Sowie dieser Thread:

https://www.bsdforen.de/threads/was-fuer-einen-chipset-hat-die-tp-link-tg-3468.22829/

Der RTL8169 scheint im Gegensatz zum RTL8111 kein RSS zu unterstützen, was die Last einer Verbindung auf mehrere Kerne verteilt. Hast du einen 2-Kern oder 4-Kern i7?

Du könntest also mal einen RTL8111 testen oder direkt eine Intel Karte.

https://www.intel.de/content/www/de/de/support/articles/000006703/ethernet-products.html

https://www.heise.de/preisvergleich/3050785251
https://www.heise.de/preisvergleich/inter-tech-argus-st-729-gigabit-lan-adapter-77773003-a2916365.html
https://www.heise.de/preisvergleich/inter-tech-argus-st-727-77773002-a2544895.html

Würde die Intel I210 nehmen, die ist sehr bewährt.

Laut diesem Tutorial steht in der Client-Konfiguration bei "[Interface]" unter "Address" ebenfalls die IP vom Server und nicht vom Client.

https://blog.francium.tech/wireguard-vpn-server-and-client-configuration-abe8a18e8192

Nach etwas Recherche in der strongSwan Dokumentation bin ich auf MOBIKE gestoßen.

https://docs.strongswan.org/docs/5.9/features/mobike.html

Ein Protokoll, welches eine nahtlose Veränderung der öffentlichen IPs erlaubt, also genau das was ich benötige für die Redundanz.

Leider wird das erst ab IKEv2 unterstützt und die Sophos UTM wurde leider nicht mehr auf IKEv2 aktualisiert, damit sie kein Konkurrent für die neue SFOS Firewall ist. Wäre auch sinnvoll die MOBIKE Kontrollbox auszublenden wenn in der Dropdown-Liste IKEv1 ausgewählt wird.

Von UTM zu UTM scheint das auch mit IKEv1 zu funktionieren, da die UTM das wohl außerhalb von strongSwan über 2 Tunnel mit "Uplink Balancing" löst.

https://community.sophos.com/utm-firewall/f/german-forum/115525/sophos-utm-multiple-s2s-ipsec-vpn-mit-failover-tutorial-de

Die Lösung funktioniert aber nicht über Kreuz, sondern jeweils mit 2 feste Paaren.

Das hatten schon mehrere Leute versucht:

https://github.com/opnsense/core/issues/1697
https://forum.opnsense.org/index.php?topic=7350.0

Aber es scheitert wohl daran, dass du nur CA-Zertifikate angeben kannst und keine gewöhnlichen Server-Zertifikate. Und CA-Zertifikate erstellt z.B. Let's Encrypt nur für sich selbst.

Soweit ich verstanden habe, erstellt der Webproxy (Squid) durch die CA dynamisch Zertifikate um die Domain von der tatsächlich angefragten Webseite zu imitieren.

https://wiki.squid-cache.org/Features/BumpSslServerFirst