Messages

31

General Discussion / Re: LACP is not working

« on: November 25, 2020, 07:07:18 pm »

i have been doing some dugging on the switch and found out this.

the only thing i see its mismatch error.

Code: [Select]

Dynamic Log Buffer (50 lines):
Nov 26 02:29:40:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:40:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:40:I:Trunk: Group (1/1/1, 2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:40:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:40:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:40:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:39:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is up.
Nov 26 02:29:39:I:Trunk: Group (1/1/1, 2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:33:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:33:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:33:I:Trunk: Group (1/1/1, 2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:33:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:33:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:33:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:32:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is up.
Nov 26 02:29:32:I:Trunk: Group (1/1/1, 2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:26:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:26:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:26:I:Trunk: Group (1/1/1, 2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:26:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:26:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:26:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:25:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is up.
Nov 26 02:29:25:I:Trunk: Group (1/1/1, 2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:19:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:19:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:19:I:Trunk: Group (1/1/1, 2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:19:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:19:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:19:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:18:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is up.
Nov 26 02:29:18:I:Trunk: Group (1/1/1, 2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:12:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:12:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:12:I:Trunk: Group (1/1/1, 2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:12:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:12:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is down.
Nov 26 02:29:12:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:11:I:System: Logical link on dynamic lag interface ethernet 1/1/1 is up.
Nov 26 02:29:11:I:System: Interface ethernet 1/1/1, state up
Nov 26 02:29:11:I:Trunk: Group (1/1/1, 2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:05:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:05:I:Trunk: Group (2/1/1) removed by 802.3ad link-aggregation module.
Nov 26 02:29:05:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is down.
Nov 26 02:29:05:I:System: dynamic lag interface 2/1/1's peer info (priority=3,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:05:I:System: dynamic lag interface 1/1/1's peer info (priority=4,id=e839.3511.faab,key=0) mis-matches with lag's peer info (priority=32768,id=e839.3511.faab,key=715), set to mismatch Error
Nov 26 02:29:04:I:System: Logical link on dynamic lag interface ethernet 2/1/1 is up.
Nov 26 02:29:04:I:System: Interface ethernet 2/1/1, state up
Nov 26 02:29:01:I:Trunk: Group (2/1/1) created by 802.3ad link-aggregation module.
Nov 26 02:29:01:I:System: dynamic lag 1, has new peer info (priority=32768,id=e839.3511.faab,key=715) (N/A)

32

General Discussion / Re: LACP is not working

« on: November 24, 2020, 04:17:38 pm »

What's a timout? Sorry I have not configured anything like that. The Cisco config is simply:

Code: [Select]

interface Port-channel4
 description OPNsense
 switchport mode trunk
!
interface GigabitEthernet0/15
 description OPNsense
 switchport mode trunk
 channel-group 4 mode active
!
interface GigabitEthernet0/16
 description OPNsense
 switchport mode trunk
 channel-group 4 mode active

And that's that. Works as intended.

 i just tried the configuration with unifi layer 3 switch and it does works.
this works with single switch as lag, i dont know if the issue is related to the stack or something else.

33

General Discussion / Re: LACP is not working

« on: November 24, 2020, 01:13:43 pm »

@Pmhausen thank you so much for your answer.
i hope someone will advise about the hash-lag

Code: [Select]

Trunk Type:    hash-based
i've been doing some digging too i, i have figure out a difference on the time out of the LAGG

Long LACP timeouts (the TIO flag) on the switch is short, is yours long or also short on the LACP?

this a working confi from a cisco i tested today

Code: [Select]

LAG Configuration:
   Ports:         e 1/2/1 e 1/2/3
   Port Count:    2
   Primary Port:  1/2/1
   Trunk Type:    hash-based
   LACP Key:      22047
   [b]LACP Timeout:  long[/b]

34

General Discussion / Re: LACP is not working

« on: November 24, 2020, 12:19:59 am »

Did I miss a copy of ifconfig lagg0 on the OPNsense side or did you not provide one, yet?
Please do so.

Thank you, this what the lag config shows.

Code: [Select]

root@firewall:~ # ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=850098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO>
        ether e8:39:35:11:fa:ab
        inet6 fe80::ea39:35ff:fe11:faab%lagg0 prefixlen 64 scopeid 0xb
        inet 192.168.55.1 netmask 0xffffff00 broadcast 192.168.55.255
        laggproto lacp lagghash l2,l3,l4
        laggport: em2 flags=8<COLLECTING>
        laggport: em3 flags=8<COLLECTING>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

on the switch side it shows

Code: [Select]

LAG Configuration:
   Ports:         e 1/1/2 e 2/1/2
   Port Count:    2
   Primary Port:  1/1/2
   Trunk Type:    hash-based
   LACP Key:      20002
Deployment: HW Trunk ID 2
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/2      Up      Blocked Full 1G    2     Yes N/A  0   609c.9f4b.808d  LAN1
2/1/2      Up      Blocked Full 1G    2     Yes N/A  0   609c.9f4b.808d  LAN2

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/1/2           1        1   20002   Yes   S   Agg  No   No   No   No   No   Ina
2/1/2           1        1   20002   Yes   S   Agg  Syn  No   No   Def  Exp  Err
and

Code: [Select]

clog /var/log/system.log

Code: [Select]

Nov 24 00:13:35 firewall kernel: em2: lacpdu receive
Nov 24 00:13:35 firewall kernel: actor=(0001,60-9C-9F-4B-80-8C,4E22,0001,0002)
Nov 24 00:13:35 firewall kernel: actor.state=7<ACTIVITY,TIMEOUT,AGGREGATION>
Nov 24 00:13:35 firewall kernel: partner=(8000,E8-39-35-11-FA-AB,016B,8000,0003)
Nov 24 00:13:35 firewall kernel: partner.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
Nov 24 00:13:35 firewall kernel: maxdelay=0
Nov 24 00:13:35 firewall kernel: em2: old pstate cf<ACTIVITY,TIMEOUT,AGGREGATION,SYNC,DEFAULTED,EXPIRED>
Nov 24 00:13:35 firewall kernel: em2: new pstate f<ACTIVITY,TIMEOUT,AGGREGATION,SYNC>
Nov 24 00:13:35 firewall kernel: em3: lacpdu transmit
Nov 24 00:13:35 firewall kernel: actor=(8000,E8-39-35-11-FA-AB,016B,8000,0004)
Nov 24 00:13:35 firewall kernel: actor.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
Nov 24 00:13:35 firewall kernel: partner=(0001,60-9C-9F-4B-80-8C,4E22,0001,0102)
Nov 24 00:13:35 firewall kernel: partner.state=cf<ACTIVITY,TIMEOUT,AGGREGATION,SYNC,DEFAULTED,EXPIRED>
Nov 24 00:13:35 firewall kernel: maxdelay=0
Nov 24 00:13:35 firewall kernel: em2: lacpdu transmit
Nov 24 00:13:35 firewall kernel: actor=(8000,E8-39-35-11-FA-AB,016B,8000,0003)
Nov 24 00:13:35 firewall kernel: actor.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
Nov 24 00:13:35 firewall kernel: partner=(0001,60-9C-9F-4B-80-8C,4E22,0001,0002)
Nov 24 00:13:35 firewall kernel: partner.state=f<ACTIVITY,TIMEOUT,AGGREGATION,SYNC>
Nov 24 00:13:35 firewall kernel: maxdelay=0

35

General Discussion / Re: LACP is not working

« on: November 23, 2020, 09:51:45 pm »

You really should read about switch Troubleshooting, they get way more into detail

actually I do  the switch shows

Code: [Select]

=== LAG "OPNSENSEWAN" ID 1 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/2/1 e 1/2/1
   Port Count:    2
   Primary Port:  1/2/1
   Trunk Type:    hash-based
   LACP Key:      20001
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/2/1      Up      Blocked Full 1G    1     Yes 18   0   609c.9f3a.a488 
1/2/2      Up      Blocked  Full 1G   1     Yes 18   0   609c.9f3a.a488

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/1          1        1   20001   Yes   S   Agg  Syn  Col  Dis  Def  No   Ina
1/2/2.         1        1   20001   Yes   S   Agg  Syn  No   No   Def  No   Ina

Code: [Select]

Just know that if you get the above message (GigabitEthernet x/x/x is up, line protocol is down (LACP-BLOCKED), you have a LAG protocol mismatch.. this what the support teams says.

mean OPNsense is not sending LACPDU's (802.3ad) to the switch, and the switch cannot so the switch cannot breng the Brundle online.

how I can check if the LAG on the opnsense is already sending 802.3ad.

on the Opnsense LAG Protocol shows LCAP is

Code: [Select]

• lacp
• Supports the IEEE 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups. Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports. In the event of changes in physical connectivity, Link Aggregation will quickly converge to a new configuration.

36

General Discussion / Re: LACP is not working

« on: November 23, 2020, 08:43:12 pm »

@pmhausen today I have tried it but when I connect the cables the below has showen up.

Code: [Select]

lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
interface we are using on the box are

Code: [Select]

Intel(R) PRO/1000 Network Connection
the Possible Flapping is the client is not sending fact 'laggproto lacp' wasn't set on the client-side.
but the configuration is already done on the LAG to use LACP.

after I rebooted the box I get those logs

Code: [Select]

WARNING: /mnt was not properly dismounted
WARNING: /mnt: mount pending error: blocks 259880 files 7
ugen0.2: <vendor 0x8087 product 0x0024> at usbus0
uhub2 on uhub0
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
ugen1.2: <vendor 0x8087 product 0x0024> at usbus1
uhub3 on uhub1
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
WARNING: /mnt: reload pending error: blocks 259880 files 7
uhub2: 6 ports with 6 removable, self powered
uhub3: 6 ports with 6 removable, self powered
ugen0.3: <Avocent USB Composite Device-0> at usbus0
kbd2 at ukbd1
ugen1.5: <vendor 0x192f USB Optical Mouse> at usbus1
WARNING: /mnt: reload pending error: blocks 259880 files 7
lagg0: IPv6 addresses on em2 have been removed before adding it as a member to prevent IPv6 address scope violation.
lagg0: link state changed to DOWN
em3: link state changed to DOWN
lagg0: IPv6 addresses on em3 have been removed before adding it as a member to prevent IPv6 address scope violation.
em0: link state changed to UP
em0_vlan20: link state changed to UP
em0_vlan40: link state changed to UP
em0_vlan10: link state changed to UP
em0_vlan11: link state changed to UP
em0_vlan12: link state changed to UP
em0_vlan13: link state changed to UP
em0_vlan30: link state changed to UP
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
lagg0: link state changed to UP
em3: Interface stopped DISTRIBUTING, possible flapping
em2: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
em2 and em3 are the interfaces on the LAG.

37

General Discussion / Re: LACP is not working

« on: November 23, 2020, 12:31:01 pm »

Do you know where i can see the logs of LAAG on the OPNsens?
i have used this command

Code: [Select]

sysctl net.link.lagg.lacp.debug=1
disconnected and connected the cables but nothing shows on the console.

38

General Discussion / Re: LACP is not working

« on: November 22, 2020, 10:05:23 pm »

I’ll will check tomorrow as first thing when I get to the office
I appreciate it really
I am worried when it fixed and remotely update the box will cause the same failure
But will have to fix it first

Thank you

39

General Discussion / Re: LACP is not working

« on: November 22, 2020, 06:22:39 pm »

We all are just guessing without debug logs

Does OPNsens has this option to check the LACP debug? like on pfsense I run this

Code: [Select]

sysctl net.link.lagg.lacp.debug=1
I am not near the switch and the box just collecting the info to do tomorrow.

I've been doing some reading I understand it could be related to strict mode on opnsense is different than pfsense.
 pfsense use 0 and opnsense use 1 ? is this correct ?

Code: [Select]

sysctl net.link.lagg.0.lacp.lacp_strict_mode=0

40

General Discussion / Re: LACP is not working

« on: November 22, 2020, 05:32:08 pm »

I can only comment that lagg does work in the general case.

My OPNsense:

Code: [Select]

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC>
ether 00:0d:b9:57:27:90
inet6 fe80::20d:b9ff:fe57:2790%lagg0 prefixlen 64 scopeid 0x9
laggproto lacp lagghash l2,l3,l4
laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

My Cisco 2960-L:

Code: [Select]

cisco#sh lacp 4 neighbor
Flags:  S - Device is requesting Slow LACPDUs
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode     

Channel group 4 neighbors

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi0/15    SA      32768     000d.b957.2790   5s    0x0    0x12B  0x1     0x3D 
Gi0/16    SA      32768     000d.b957.2790   4s    0x0    0x12B  0x2     0x3D 

So, does your Brocade switch have some debugging capability? E.g. if I bring one of my two links down on the OPNsense side, enable debugging of LACP events on the Cisco, then bring the interface up again, I get this:

Code: [Select]

cisco#debug lacp event
Link Aggregation Control Protocol events debugging is on
cisco#
Nov 22 11:48:29.174: LACP: Gi0/16 set to UNSELECTED
Nov 22 11:48:30.170: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/16, changed state to down
Nov 22 11:48:31.174: %LINK-3-UPDOWN: Interface GigabitEthernet0/16, changed state to down
Nov 22 11:48:33.939: %LINK-3-UPDOWN: Interface GigabitEthernet0/16, changed state to up
Nov 22 11:48:34.943: LACP: Gi0/16 STANDBY aggregator hex address is 64DA810
Nov 22 11:48:34.944: LACP: Gi0/16 set to STANDBY
Nov 22 11:48:36.722: lacp_handle_standby_port_internal called, depth = 1
Nov 22 11:48:36.722: LACP: Gi0/16 standby->selected
Nov 22 11:48:36.722: LACP: Gi0/16 set to SELECTED
Nov 22 11:48:38.551: lacp_handle_standby_port_internal called, depth = 1
Nov 22 11:48:39.551: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/16, changed state to up

Please try and find some more detailled information on the switch side.

Kind regards,
Patrick

I really appreciate your answer, I don't know if the switch has debug as I come from Cisco. what the crazy is when I connect a synolougy to the same ports its detect the LACP/ Pfsense Does.
seems like the MAC address of the NiCS somehow holded by the switch and cause the spanning tree act crazy?
is this possible?

41

General Discussion / Re: LACP is not working

« on: November 22, 2020, 12:46:50 pm »

Same author? 

Maybe FreeBSD 12 is handling lacp bit different. Then you should have the same Error when Testing pfsense 2.5

Thank you for your answer. on the switch I can the LAG went to block when I connect the cables.
isn't pfsense using the same Freebsd 12 as OPNsense?

42

General Discussion / Re: LACP is not working

« on: November 22, 2020, 12:36:31 pm »

Same author? 

Haha, sorry, didn't had a look at the authors name 

sorry contacted the support and they advise to open a new case at Generat Discussion,
so I just did

43

General Discussion / LACP is not working

« on: November 22, 2020, 12:30:48 am »

Dear all,
We have been using OPNsense for sometimes now, one our customer has two ISP switch Layer 3, each Switch provides 1GB NIC with 1Gbps speed to the opnsense
this configuration has been working excellent before with pfsense.
Switches are configured to do LACP on interfaces ( as I mentioned it been working for long ).

Switch 1 >>>> Port 1 >>>>> Pfsense Port 1 ( now is Opnsense )
Switch 2 >>>> Port 1 >>>>> Pfsense Port 2 ( now is opnsense)

Both switches are stacked and are Brocade ICX7250.

as I mentioned this config been working with pfsense for long time until we convince the customer to move to OPNsens.

the issue is:

We have created LAGG see attacked this LAG is em2 and em3 as LAN LAGG. so whenever we connect the cables to the switch the error pops on the console of the OPNS and keeps popping ups.

Code: [Select]

interface stopped distributing possible flapping

after I tried a pfsense installation clean installation and I create the LAGG everything works as expected.

Can someone please help clear things to me, is it the OPNsens? configuration ? what am I doing wrong?

appreciate any help/ideas.

44

20.7 Legacy Series / Re: 2 Switch 1 WAN

« on: November 20, 2020, 04:11:03 pm »

As you said your switches are stacked and they support LACP. So this could be one solution. What happens if one of the stack members fails? Did/can you try that?

Usually for redundancy you would use two individual switches and connect the firewall to both of them. Then you would configure a bridge with STP/RSTP including both interfaces. You’ll lose the increased bandwidth in comparison to the LACP way.

I’m not an expert on STP though, so maybe another member has more to say about that.

Today we have been trying to archive this. however it didnt works.

I have created a LAGG with two free interfaces using LACP, on the Switch we have LACP for two interfaces port 1 of switch 1 and port 1 of switch as LACP ports.

whenever we conenct the cables on the switches on the OPNs console start popping up " interface stopped distributing possible flapping"


when i connect our NAS on the same ports it works.

can someone please advies what have we done wrong?

45

Intrusion Detection and Prevention / IDS OVER VPN

« on: November 20, 2020, 02:27:56 am »

Dear All,

we have been using OPNsense for over 4 years, we have happy with it, and big thank you the developers for this great software.
I have a couple of concerns.

1- IDS/IPS enabled in the WAN.
we have IPS /IDS enabled on the WAN,i've attend a online training of suricata they advise to run the IDS/IPS on the LAN because OPNsense blocks anything on the WAN already.

we have site to site VPN from Office 1 to Office 2 when I apply the IDS/IPS on the LAN Interface I cannot connect using RDP/https/ ect... I even cannot ping.
on the IDS alert there is nothing there about those connections been blocked. when I switch to WAN stuff start working.

What am I doing wrong?

appreciate each support.