Messages

1

Intrusion Detection and Prevention / Re: Suricata, LAN and VLAN

« on: January 15, 2021, 10:49:15 pm »

For IPS and VLANs, you should enable Promiscuous Mode. I run IPS on my LAN with multiple VLANs, without any issues.

Try enabling that and see if it makes any difference.

I have exact the same problem. Remote users om vlans cannot connect after I enable the ids on the lan.
I cannot run it on LAN neither with promiscuous mode or not . It crashes my connection
I cannnot see why have to run it in the wan as the way already dropping anything not allowed.

2

20.7 Production Series / Re: Update 20.7.6 to 20.7.7 Update ERR_SSL_PROTOCOL_ERROR

« on: January 14, 2021, 11:32:15 pm »

Then it looks like lighttpd is not going to fix that issue. I can't imagine that this is an issue that can't be fixed from the system (switching cert maybe?). Because it is working for a representative amount of users...


Cheers,
Franco

I just updated both my backup and master, the master runs Let's Encrypt and I had to run:

Code: [Select]

opnsense-revert -r 20.7.6 lighttpd && configctl webgui restart
This got the GUI working again but on updating lighttpd the error came back.

Is it a case of wait for further update or is there something I can do?

Thanks,

Nick

EDIT: Proper schoolboy error and completely missed the 2nd page ignore me please

I am just curious how to do so if you cannot access the gui

Code: [Select]

Please note that Let's Encrypt users need to reissue their certificates
manually after upgrading to this version to fix the embedded certificate chain
issue with the current signing CA switch going on.

3

20.7 Production Series / Re: Update 20.7.6 to 20.7.7 Update ERR_SSL_PROTOCOL_ERROR

« on: January 07, 2021, 12:08:13 pm »

Then it looks like lighttpd is not going to fix that issue. I can't imagine that this is an issue that can't be fixed from the system (switching cert maybe?). Because it is working for a representative amount of users...


Cheers,
Franco

for me is working too in some boxes, but most of them are crashed

4

20.7 Production Series / Re: Update 20.7.6 to 20.7.7 Update ERR_SSL_PROTOCOL_ERROR

« on: January 05, 2021, 03:01:36 am »

this the fix

Code: [Select]

opnsense-revert -r 20.7.6 lighttpd && configctl webgui restart
the

Code: [Select]

opnsense-revert -r 20.7.7 lighttpd && configctl webgui restart is not working

first run the first command of 20.7.6 and access the gun and run the update from the gui to lighted 1.4.58 the errors appears again.

5

20.7 Production Series / Re: Update 20.7.6 to 20.7.7 Update ERR_SSL_PROTOCOL_ERROR

« on: December 21, 2020, 04:06:00 pm »

did the last update change something on the firewall behaivor?
i noticed our UDP packets (VOIP) are disconnecting after 20 sec. also the web gui is not availble.

6

Sensei / Re: Sensei on OPNsense - Application based filtering

« on: December 21, 2020, 02:49:09 pm »

we have users reporting issue its blocks website when they visit them for the first time.
how to fix this ?

7

General Discussion / Re: LACP is not working

« on: December 08, 2020, 12:19:56 am »

I have configured the long timeout on the LAG interface but it not showing it on the interface.
anyway I can force that on the lag interface?

Code: [Select]

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800008<VLAN_MTU>
        ether e8:39:35:11:fa:ab
        inet6 fe80::ea39:35ff:fe11:faab%lagg0 prefixlen 64 scopeid 0xb
        inet 192.168.73.1 netmask 0xffffff00 broadcast 192.168.73.255
        laggproto lacp lagghash l2,l3,l4
        laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: em3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>


8

Intrusion Detection and Prevention / Re: IDS OVER VPN

« on: December 06, 2020, 10:35:16 pm »

Do you have "Block" Rules enabled?

What rules are you referring to? On the ids site ?

Yes IDs rules

yes we do have them enable to block the IDS rules.
see attached

9

General Discussion / Re: LACP is not working

« on: December 04, 2020, 01:00:56 am »

We have contacted the switch support team and the issue appear to be on the switch and the way it deal with the LACP.
They are aware of but not planning on changing anything in the near future as the switch is out of support.

10

General Discussion / Re: LACP is not working

« on: December 02, 2020, 09:18:05 pm »

Could you run

'sysctl -A | grep *your network card driver* - i.e. 'sysctl -A | grep em or igb'

Also 'sysctl -A | grep lacp' - lets see your lacp settings

And post output of each command, em or igb is the intel driver

eee is an energy efficient feature can cause issues on freebsd for nics. By setting 'hw.em.eee_setting = 0' in the tunables will turn this off for all nics.

Also what is your lagg settings on your opnsense? - have you tried loadbalance mode?

also on opnsense, lets see the lacp debugging

'sysctl net.link.lagg.lacp.debug=1' - then share the system log - 'clog /var/log/system.log'


IMAHO: It seems someone else has the same issue as you on pfsense - https://forum.netgate.com/topic/158534/lacp-not-working/79 - and thats a brocade switch.

thank you for your answer,
this the log on t he system.log

Code: [Select]

Dec  2 17:37:17 firewall /flowd_aggregate.py[18361]: vacuum src_addr_details_086400.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum src_addr_000300.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum src_addr_003600.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum src_addr_086400.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum dst_port_000300.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum dst_port_003600.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum dst_port_086400.sqlite
Dec  2 17:37:19 firewall /flowd_aggregate.py[18361]: vacuum interface_000030.sqlite
Dec  2 17:37:20 firewall /flowd_aggregate.py[18361]: vacuum interface_000300.sqlite
Dec  2 17:37:20 firewall /flowd_aggregate.py[18361]: vacuum interface_003600.sqlite
Dec  2 17:37:20 firewall /flowd_aggregate.py[18361]: vacuum interface_086400.sqlite

Code: [Select]

root@firewall:~ # sysctl -A | grep lacp
net.link.lagg.lacp.default_strict_mode: 1
net.link.lagg.lacp.debug: 0

I am not sure what to do here

Code: [Select]

'sysctl -A | grep *your network card driver* - i.e. 'sysctl -A | grep em or igb'
do. you mean to run sysctl -A | grep em or igb01 if the interface is igb01.
or you mean I have to run it on the lag interface?

like this ?

Code: [Select]

root@firewall:~ # sysctl -A | grep em3
irq267: em3:irq0:77 @cpu0(domain0): 76231

11

General Discussion / Re: LACP is not working

« on: November 30, 2020, 03:05:17 pm »

To help along further with this, there is some possible issues occuring:

1. Have you turned of all offloading options on opnsense? - TSO, LRO? - make sure with the ifconfig
2. Port flap dampening configuration is worth looking at on this switch?
3. Energy saving features of the nics on the server - i.e. hw.em.eee_setting = 0

Maybe do a 'sysctl -A | grep *your network card driver* - 'sysctl -A | grep em or igb'

Also sysctl -A | grep lacp - lets see your lacp settings

Thank you for your answer

Code: [Select]

1. Have you turned of all offloading options on opnsense? - TSO, LRO? - make sure with the ifconfigyes i did in interface settings , Hardware CRC / Hardware TSO /Hardware LRO / VLAN Hardware Filtering    are disabled
when i do ifconfig -vv lagg0

Code: [Select]

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800008<VLAN_MTU>
        ether e8:39:35:11:fa:ab
        inet6 fe80::ea39:35ff:fe11:faab%lagg0 prefixlen 64 scopeid 0xb
        laggproto lacp lagghash l2,l3,l4
        lagg options:
                flags=10<LACP_STRICT>
                flowid_shift: 16
        lagg statistics:
                active ports: 0
                flapping: 0
        lag id: [(0000,00-00-00-00-00-00,0000,0000,0000),
                 (0000,00-00-00-00-00-00,0000,0000,0000)]
        laggport: em2 flags=0<> state=41<ACTIVITY,DEFAULTED>
                [(8000,E8-39-35-11-FA-AB,8003,8000,0003),
                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
        laggport: em3 flags=0<> state=41<ACTIVITY,DEFAULTED>
                [(8000,E8-39-35-11-FA-AB,8004,8000,0004),
                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
        groups: lagg
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Code: [Select]

2. Port flap dampening configuration is worth looking at on this switch?link error dampening is enabled

Code: [Select]

SSH@Ruckus2050-LS#show interface ethernet 1/1/2
GigabitEthernet1/1/2 is up, line protocol is down (LACP-BLOCKED)
  Port down (LACP-BLOCKED) for 1 day(s) 14 hour(s) 28 minute(s) 40 second(s)
  Hardware is GigabitEthernet, address is 609c.9f4b.808d (bia 609c.9f4b.808d)
  Configured speed auto, actual 1Gbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual MDIX
  EEE Feature Disabled
  Member of 7 L2 VLANs, port is tagged, port state is BLOCKING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Enabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is config enabled, oper enabled, negotiation disabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  Member of active trunk ports 1/1/2,2/1/2, primary port is 1/1/2
  Member of configured trunk ports 1/1/2,2/1/2, primary port is 1/1/2
  Port name is LAN1
  IPG MII 96 bits-time, IPG GMII 96 bits-time
  MTU 10200 bytes, encapsulation ethernet
  300 second input rate: 0 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 928 bits/sec, 0 packets/sec, 0.00% utilization
  15187 packets input, 1943872 bytes, 0 no buffer
  Received 1 broadcasts, 15186 multicasts, 0 unicasts
  0 input errors, 0 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  154231 packets output, 19755504 bytes, 0 underruns
  Transmitted 214 broadcasts, 153930 multicasts, 86 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled

UC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0                   0                   0
         1                   0                   0
         2                   0                   0
         3                   0                   0
         4                   0                   0
         5                   0                   0
         6                   0                   0
         7              152356                   0


MC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0                 217                   0
         1                 148                   0
         2                1510                   0
         3                   0                   0

Code: [Select]

3. Energy saving features of the nics on the server - i.e. hw.em.eee_setting = 0
i am not sure i understand this correctly?

I see also MAC- learning is enabled, we have two opnsense boxes connected to the switch one is on and one is off, both boxes run the same configuration in case first one goes down we fired up the second one, maybe the Mac-learning causing this?

12

General Discussion / Re: LACP is not working

« on: November 28, 2020, 10:27:28 pm »

edit: unfrotunately its still not working

I'm really sorry to hear that.

Options:

• go with pfSense 2.4.x but you may run into the problem again when 2.5 is released. So maybe it's worth to try the 2.5 beta first. If it has the same problem it's BSD 12 related.
  
• try OPNsense 20.1 and wait if there will be a patch someday
  
• try different switches
  
• are you able to replace the network cards in that server? Maybe it's only related to some drivers.
  

thank you so much for your continue answers. i'll reach out to the support and see what is the cause.

the server has two different NICS i tried them both same error.

appreciate it really


Edit.

i noticed two LAGS are sharing the same MAC address is this maybe the cause?

Code: [Select]

Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/1      Down    None    None None  1     No  141  0   609c.9f4b.808c  WAN1
2/1/1      Down    None    None None  1     No  141  0   609c.9f4b.808c  WAN2

Code: [Select]

Deployment: HW Trunk ID 3
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/11     Up      Forward Full 1G    11    Yes 141  0   609c.9f4b.808c
1/1/12     Up      Forward Full 1G    11    Yes 141  0   609c.9f4b.808c
2/1/11     Up      Forward Full 1G    11    Yes 141  0   609c.9f4b.808c
2/1/12     Up      Forward Full 1G    11    Yes 141  0   609c.9f4b.808c

13

General Discussion / Re: LACP is not working

« on: November 28, 2020, 10:06:35 pm »

Interfaces: Settings

VLAN Hardware Filtering. Set it to disable.

And all other HW stuff should be disabled (checkboxes checked), too.

It's a global setting for all interfaces.

thank you, i just disabled it now.
it was enabled : Enable VLAN hardware Filtering now its disabled.
ill see if i can get the LACP working now
 will report back in a min


edit: unfrotunately its still not working

Code: [Select]

lagg0: link state changed to UP
bce1: Interface stopped DISTRIBUTING, possible flapping
bce0: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
bce1: Interface stopped DISTRIBUTING, possible flapping
bce0: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
bce1: Interface stopped DISTRIBUTING, possible flapping
bce0: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN
lagg0: link state changed to UP
bce1: Interface stopped DISTRIBUTING, possible flapping
bce0: Interface stopped DISTRIBUTING, possible flapping
lagg0: link state changed to DOWN

14

General Discussion / Re: LACP is not working

« on: November 28, 2020, 09:59:19 pm »

thank you for your answer.
i am not near the box have to drive 2hr to get there.
is this a bug/issue with 20.7?
just trying to understand to plan my next move.

It may be related to BSD12. So even if you go for pfSense 2.4.x now, it may break with 2.5.
That's the reason why I asked if it's working with OPNsense 20.1 or pfSense 2.5

It could be a combination of BSD12 and your switches, because other people say it's working for them.

Just found this:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241785

Is this describing your issue?

thank you for your answer.
yes its my issue. exactly as showen.
where do they mean to disable those settings?
 

Code: [Select]

We also use vlan + lagg + ix and we often need to add/remove vlans, so as a temporary solution we disable vlanhwfilter on lagg interface.
i just tried it nowhome with a unifi switch and it works. same settings.

Code: [Select]

OPNsense 20.7.5-amd64
FreeBSD 12.1-RELEASE-p10-HBSD
OpenSSL 1.1.1h 22 Sep 2020

Code: [Select]

Nov 28 21:52:08 firewall kernel: igb5: lacpdu transmit
Nov 28 21:52:08 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:08 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:08 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: maxdelay=0
Nov 28 21:52:08 firewall kernel: igb4: lacpdu transmit
Nov 28 21:52:08 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:08 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:08 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: maxdelay=0
Nov 28 21:52:16 firewall kernel: igb5: lacpdu receive
Nov 28 21:52:16 firewall kernel: actor=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:16 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: partner=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:16 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: maxdelay=0
Nov 28 21:52:16 firewall kernel: igb4: lacpdu receive
Nov 28 21:52:16 firewall kernel: actor=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:16 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: partner=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:16 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: maxdelay=0
 Nov 28 21:52:38 firewall kernel: igb5: lacpdu transmit
Nov 28 21:52:38 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:38 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:38 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: maxdelay=0
Nov 28 21:52:38 firewall kernel: igb4: lacpdu transmit
Nov 28 21:52:38 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:38 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:38 firewall kernel:

15

General Discussion / Re: LACP is not working

« on: November 28, 2020, 08:59:43 pm »

Reinstall 20.1 and restore config

thank you for your answer.
i am not near the box have to drive 2hr to get there.
is this a bug/issue with 20.7?
just trying to understand to plan my next move.


i have fired up a hardware home with unifi switch, i ve created LACP and it seems to work.

this working with this release

Code: [Select]

OPNsense 20.7.5-amd64
FreeBSD 12.1-RELEASE-p10-HBSD
OpenSSL 1.1.1h 22 Sep 2020

Code: [Select]

Nov 28 21:52:08 firewall kernel: igb5: lacpdu transmit
Nov 28 21:52:08 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:08 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:08 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: maxdelay=0
Nov 28 21:52:08 firewall kernel: igb4: lacpdu transmit
Nov 28 21:52:08 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:08 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:08 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:08 firewall kernel: maxdelay=0
Nov 28 21:52:16 firewall kernel: igb5: lacpdu receive
Nov 28 21:52:16 firewall kernel: actor=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:16 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: partner=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:16 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: maxdelay=0
Nov 28 21:52:16 firewall kernel: igb4: lacpdu receive
Nov 28 21:52:16 firewall kernel: actor=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:16 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: partner=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:16 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:16 firewall kernel: maxdelay=0
 Nov 28 21:52:38 firewall kernel: igb5: lacpdu transmit
Nov 28 21:52:38 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0006)
Nov 28 21:52:38 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0018)
Nov 28 21:52:38 firewall kernel: partner.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: maxdelay=0
Nov 28 21:52:38 firewall kernel: igb4: lacpdu transmit
Nov 28 21:52:38 firewall kernel: actor=(8000,00-08-A2-0C-99-7B,020B,8000,0005)
Nov 28 21:52:38 firewall kernel: actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
Nov 28 21:52:38 firewall kernel: partner=(8000,74-83-C2-48-2F-67,0042,0080,0017)
Nov 28 21:52:38 firewall kernel: