Hi everyone!
Hope you have a great day.
I have a Netgate 8200 flashed with OPNsense and updated to OPNsense 25.1.7_4.
Layout of Networking Ports: https://docs.netgate.com/pfsense/en/latest/solutions/netgate-8200/io-ports.html#networking-ports
Goal is to create a lacp lagg consiting of ix0 and ix1 with 4 VLANs ontop towards a HP 6300M stack.
ix0 is already connected and ix1 will be connected soon. To have everything ready and tested it is a one port lagg at the moment (I also tested it with both Ports, same problem)
ix0 is up:
ifconfig -vvvv ix0
ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48538b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 90:ec:77:8f:7c:b7
media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: ix0
plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
vendor: Intel Corp PN: E10GSFPSR-L SN: XXXXX DATE: 2024-08-13
module temperature: 41.61 C voltage: 3.29 Volts
lane 1: RX power: 0.51 mW (-2.96 dBm) TX bias: 5.99 mA
SFF8472 DUMP (0xA0 0..127 range):
03 04 07 10 00 00 01 00 00 00 00 06 67 02 00 00
08 03 00 1e 49 6e 74 65 6c 20 43 6f 72 70 20 20
20 20 20 20 00 00 1b 21 45 31 30 47 53 46 50 53
52 2d 4c 20 20 20 20 20 41 20 20 20 03 52 00 ed
00 3a 00 00 55 53 58 32 30 30 30 30 30 31 32 37
32 36 30 30 32 34 30 38 31 33 30 30 68 fa 03 b5
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
lagg1 is up but show active_ports: 0:
ifconfig -vvvv lagg1
lagg1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48538b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
ether 90:ec:77:8f:7c:b7
hwaddr 00:00:00:00:00:00
laggproto lacp lagghash l2,l3,l4
lagg options:
flags=0<>
flowid_shift: 16
lagg statistics:
active ports: 0
flapping: 0
lag id: [(0000,00-00-00-00-00-00,0000,0000,0000),
(0000,00-00-00-00-00-00,0000,0000,0000)]
laggport: ix0 flags=0<> state=45<ACTIVITY,AGGREGATION,DEFAULTED>
[(8000,90-EC-77-8F-7C-B7,01D2,8000,0005),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: lagg1
lagg1_vlan801 is up as well:
ifconfig -vvvv lagg1_vlan801
lagg1_vlan801: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: XXXX (opt1)
options=4000000<MEXTPG>
ether 90:ec:77:8f:7c:b7
inet 10.10.1.252 netmask 0xffffff00 broadcast 10.10.1.255
groups: vlan
vlan: 801 vlanproto: 802.1q vlanpcp: 0 parent interface: lagg1
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
drivername: vlan1
HP 6300M config:
interface vlan 801
ip address 10.10.1.124/24
exit
interface lag 1
no shutdown
no routing
vlan trunk allowed 801-804
exit
interface 1/1/1
lag 1
exit
show interface lag 1
Aggregate lag1 is up
Admin state is up
Description : XXXX
MAC Address : 0c:97:5f:b1:8c:00
Aggregated-interfaces : 1/1/1
Aggregation-key : 1
Speed : 10000 Mb/s
qos trust none
VLAN Mode: native-untagged
Native VLAN: 1
Allowed VLAN List: 801-804
L3 Counters: Rx Disabled, Tx Disabled
Statistic RX TX Total
---------------- -------------------- -------------------- --------------------
Packets 10533 155023 165556
Unicast 2516 2532 5048
Multicast 8002 141537 149539
Broadcast 15 10954 10969
Bytes 1237284 19306328 20543612
Jumbos 0 0 0
Dropped 0 0 0
Pause Frames 0 0 0
Errors 0 0 0
CRC/FCS 0 n/a 0
Collision n/a 0 0
Runts 0 n/a 0
Giants 0 n/a 0
show interface 1/1/1
Interface 1/1/1 is up
Admin state is up
Link state: up for 18 minutes (since Thu May 29 17:21:34 UTC 2025)
Link transitions: 117
Description: XXXX
Persona:
Hardware: Ethernet, MAC Address: 0c:97:5f:b1:8c:40
MTU 1500
Type 10G-SR / 10G SFP+ SR
Full-duplex
qos trust none
Speed 10000 Mb/s
Auto-negotiation is off
Flow-control: off
Error-control: off
Rate collection interval: 300 seconds
Rate RX TX Total (RX+TX)
---------------- -------------------- -------------------- --------------------
Mbits / sec 0.00 0.00 0.00
KPkts / sec 0.00 0.00 0.00
Unicast 0.00 0.00 0.00
Multicast 0.00 0.00 0.00
Broadcast 0.00 0.00 0.00
Utilization % 0.00 0.00 0.00
Statistic RX TX Total
---------------- -------------------- -------------------- --------------------
Packets 10532 155002 165534
Unicast 2516 2532 5048
Multicast 8001 141518 149519
Broadcast 15 10952 10967
Bytes 1237156 19303743 20540899
Jumbos 0 0 0
Dropped 0 0 0
Pause Frames 0 0 0
Errors 0 0 0
CRC/FCS 0 n/a 0
Collision n/a 0 0
Runts 0 n/a 0
Giants 0 n/a 0
Now the Problem:
ping 10.10.1.124
PING 10.10.1.124 (10.10.1.124): 56 data bytes
ping: sendto: Network is down
ping: sendto: Network is down
ping: sendto: Network is down
^C
--- 10.10.1.124 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss
There is no working network connection. Similar setup without the lagg, just the VLANS and the same network Port works fine. So this is either the lagg or the combination of lagg and vlan. Without vlans it is not working.
Hope someone can help, thanks in advance!
What kind of switch is at the other end and are the ports properly configured for LACP?
Quote from: Patrick M. Hausen on May 29, 2025, 07:46:49 PMWhat kind of switch is at the other end and are the ports properly configured for LACP?
Thanks for your answer.
Quote from: ExMKgYFb5X on May 29, 2025, 07:45:13 PMGoal is to create a lacp lagg consiting of ix0 and ix1 with 4 VLANs ontop towards a HP 6300M stack.
Quote from: ExMKgYFb5X on May 29, 2025, 07:45:13 PMHP 6300M config:
interface vlan 801
ip address 10.10.1.124/24
exit
interface lag 1
no shutdown
no routing
vlan trunk allowed 801-804
exit
interface 1/1/1
lag 1
exit
Same lag config on this switch works fine for a few Proxmox VE Nodes and is based on the HP/Aruba docs.
OK, hash policy needs to match, LACP slow or fast interval needs to match. You probably need to explicitly set these on the OPNsense side.
Most common is L2+L3 hash.
Quote from: Patrick M. Hausen on May 29, 2025, 07:57:00 PMOK, hash policy needs to match, LACP slow or fast interval needs to match. You probably need to explicitly set these on the OPNsense side.
Most common is L2+L3 hash.
Thanks for your answer.
l3-src-dst is the default hashing algo of the HP 6300M switch and therefore I configured OPNsense to use HASH Layers: L3 now
LACP rate is slow by default therefore I left Fast timeout disabled for the lagg
Unfortunately the network is still not working (I rebooted the Firewall after the changes)