Netgate 8200 with OPNsense - lagg not working

Started by ExMKgYFb5X, May 29, 2025, 07:45:13 PM

Previous topic - Next topic
Print
Go Down Pages1
User actions
May 29, 2025, 07:45:13 PM Last Edit: May 29, 2025, 07:53:57 PM by ExMKgYFb5X
Hi everyone!

Hope you have a great day.

I have a Netgate 8200 flashed with OPNsense and updated to OPNsense 25.1.7_4.

Layout of Networking Ports: https://docs.netgate.com/pfsense/en/latest/solutions/netgate-8200/io-ports.html#networking-ports

Goal is to create a lacp lagg consiting of ix0 and ix1 with 4 VLANs ontop towards a HP 6300M stack.

ix0 is already connected and ix1 will be connected soon. To have everything ready and tested it is a one port lagg at the moment (I also tested it with both Ports, same problem)

ix0 is up:
Code Select
ifconfig -vvvv ix0
ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48538b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 90:ec:77:8f:7c:b7
media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: ix0
plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
vendor: Intel Corp PN: E10GSFPSR-L SN: XXXXX DATE: 2024-08-13
module temperature: 41.61 C voltage: 3.29 Volts
lane 1: RX power: 0.51 mW (-2.96 dBm) TX bias: 5.99 mA

SFF8472 DUMP (0xA0 0..127 range):
03 04 07 10 00 00 01 00 00 00 00 06 67 02 00 00
08 03 00 1e 49 6e 74 65 6c 20 43 6f 72 70 20 20
20 20 20 20 00 00 1b 21 45 31 30 47 53 46 50 53
52 2d 4c 20 20 20 20 20 41 20 20 20 03 52 00 ed
00 3a 00 00 55 53 58 32 30 30 30 30 30 31 32 37
32 36 30 30 32 34 30 38 31 33 30 30 68 fa 03 b5
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

lagg1 is up but show active_ports: 0:

Code Select
ifconfig -vvvv lagg1
lagg1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48538b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 90:ec:77:8f:7c:b7
hwaddr 00:00:00:00:00:00
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=0<>
flowid_shift: 16
lagg statistics:
active ports: 0
flapping: 0
lag id: [(0000,00-00-00-00-00-00,0000,0000,0000),
(0000,00-00-00-00-00-00,0000,0000,0000)]
laggport: ix0 flags=0<> state=45<ACTIVITY,AGGREGATION,DEFAULTED>
[(8000,90-EC-77-8F-7C-B7,01D2,8000,0005),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: lagg1

lagg1_vlan801 is up as well:

Code Select
ifconfig -vvvv lagg1_vlan801
lagg1_vlan801: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: XXXX (opt1)
options=4000000<MEXTPG>
ether 90:ec:77:8f:7c:b7
inet 10.10.1.252 netmask 0xffffff00 broadcast 10.10.1.255
groups: vlan
vlan: 801 vlanproto: 802.1q vlanpcp: 0 parent interface: lagg1
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: vlan1

HP 6300M config:

Code Select
interface vlan 801
ip address 10.10.1.124/24
exit

interface lag 1
no shutdown
no routing
vlan trunk allowed 801-804
exit

interface 1/1/1
lag 1
exit

Code Select
show interface lag 1

Aggregate lag1 is up
 Admin state is up
 Description : XXXX
 MAC Address                 : 0c:97:5f:b1:8c:00
 Aggregated-interfaces       : 1/1/1
 Aggregation-key             : 1
 Speed                       : 10000 Mb/s
 qos trust none
 VLAN Mode: native-untagged
 Native VLAN: 1
 Allowed VLAN List: 801-804
 L3 Counters: Rx Disabled, Tx Disabled

 Statistic                          RX                   TX                Total
 ---------------- -------------------- -------------------- --------------------
 Packets                         10533               155023               165556
   Unicast                        2516                 2532                 5048
   Multicast                      8002               141537               149539
   Broadcast                        15                10954                10969
 Bytes                         1237284             19306328             20543612
 Jumbos                              0                    0                    0
 Dropped                             0                    0                    0
 Pause Frames                        0                    0                    0
 Errors                              0                    0                    0
   CRC/FCS                           0                  n/a                    0
   Collision                       n/a                    0                    0
   Runts                             0                  n/a                    0
   Giants                            0                  n/a                    0



Code Select
show interface 1/1/1

Interface 1/1/1 is up
 Admin state is up
 Link state: up for 18 minutes (since Thu May 29 17:21:34 UTC 2025)
 Link transitions: 117
 Description: XXXX
 Persona:
 Hardware: Ethernet, MAC Address: 0c:97:5f:b1:8c:40
 MTU 1500
 Type 10G-SR / 10G SFP+ SR
 Full-duplex
 qos trust none
 Speed 10000 Mb/s
 Auto-negotiation is off
 Flow-control: off
 Error-control: off
 Rate collection interval: 300 seconds

 Rate                               RX                   TX        Total (RX+TX)
 ---------------- -------------------- -------------------- --------------------
 Mbits / sec                      0.00                 0.00                 0.00
 KPkts / sec                      0.00                 0.00                 0.00
   Unicast                        0.00                 0.00                 0.00
   Multicast                      0.00                 0.00                 0.00
   Broadcast                      0.00                 0.00                 0.00
 Utilization %                    0.00                 0.00                 0.00

 Statistic                          RX                   TX                Total
 ---------------- -------------------- -------------------- --------------------
 Packets                         10532               155002               165534
   Unicast                        2516                 2532                 5048
   Multicast                      8001               141518               149519
   Broadcast                        15                10952                10967
 Bytes                         1237156             19303743             20540899
 Jumbos                              0                    0                    0
 Dropped                             0                    0                    0
 Pause Frames                        0                    0                    0
 Errors                              0                    0                    0
   CRC/FCS                           0                  n/a                    0
   Collision                       n/a                    0                    0
   Runts                             0                  n/a                    0
   Giants                            0                  n/a                    0



Now the Problem:

Code Select
ping 10.10.1.124
PING 10.10.1.124 (10.10.1.124): 56 data bytes
ping: sendto: Network is down
ping: sendto: Network is down
ping: sendto: Network is down
^C
--- 10.10.1.124 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

There is no working network connection. Similar setup without the lagg, just the VLANS and the same network Port works fine. So this is either the lagg or the combination of lagg and vlan. Without vlans it is not working.

Hope someone can help, thanks in advance!
May 29, 2025, 07:46:49 PM #1
What kind of switch is at the other end and are the ports properly configured for LACP?
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)
May 29, 2025, 07:51:56 PM #2
Quote from: Patrick M. Hausen on May 29, 2025, 07:46:49 PMWhat kind of switch is at the other end and are the ports properly configured for LACP?

Thanks for your answer.

Quote from: ExMKgYFb5X on May 29, 2025, 07:45:13 PMGoal is to create a lacp lagg consiting of ix0 and ix1 with 4 VLANs ontop towards a HP 6300M stack.

Quote from: ExMKgYFb5X on May 29, 2025, 07:45:13 PMHP 6300M config:

Code Select
interface vlan 801
ip address 10.10.1.124/24
exit

interface lag 1
no shutdown
no routing
vlan trunk allowed 801-804
exit

interface 1/1/1
lag 1
exit

Same lag config on this switch works fine for a few Proxmox VE Nodes and is based on the HP/Aruba docs.
May 29, 2025, 07:57:00 PM #3
OK, hash policy needs to match, LACP slow or fast interval needs to match. You probably need to explicitly set these on the OPNsense side.

Most common is L2+L3 hash.
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)
May 29, 2025, 08:08:49 PM #4
Quote from: Patrick M. Hausen on May 29, 2025, 07:57:00 PMOK, hash policy needs to match, LACP slow or fast interval needs to match. You probably need to explicitly set these on the OPNsense side.

Most common is L2+L3 hash.

Thanks for your answer.

l3-src-dst is the default hashing algo of the HP 6300M switch and therefore I configured OPNsense to use HASH Layers: L3 now

LACP rate is slow by default therefore I left Fast timeout disabled for the lagg

Unfortunately the network is still not working (I rebooted the Firewall after the changes)

Print
Go Up Pages1
User actions