Firewall first allow then reject 30s later

Started by bergstrom, December 16, 2022, 11:16:49 AM

Previous topic - Next topic
Print
Go Down Pages1
User actions
December 16, 2022, 11:16:49 AM Last Edit: December 16, 2022, 11:16:42 PM by bergstrom
Hi,

I have a strange problem on my lan with opnsense firewall.
From my user subnet[192.168.30.0/24] I can SSH to a cisco switch on mgmt subnet [192.168.60.0/24]. But I get disconnected after about 30-40s because the firewall suddenly rejects the traffic. I do not understand why because I have a allow any rule from (user)192.168.30.0/24 to (switch mgmt)192.168.60.0/24
Nothing changed in opnsense during the test, same src, dst and port used. I have tried to connect to different devices on the 192.168.60.0/24 subnet, same problem.
I had a ping spinning for 5 minutes without any timeouts.
I do not get this problem if I connect to the switch on a IP on same vlan as the user host.
I have built my ruleset in Firewall: Rules: Floating.
No other firewalls in use.

Versions    OPNsense 22.7.9-amd64
FreeBSD 13.1-RELEASE-p5
OpenSSL 1.1.1s 1 Nov 2022
CPU type    Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz (4 cores, 4 threads)
Memory usage 20 % ( 1658/8034 MB )

I do not have this problem on other subnets what I have seen.
IPS disabled.

screenshots attached of ruleset and log files.

Anyone know what causing this or can help me?
Thanks

Regards
December 16, 2022, 04:26:31 PM #1
Hi
any other routes between .30 and .60 subnets?
looks like tcp.opening timeout imho
December 16, 2022, 10:52:32 PM #2
Hi,
thank you for the reply.
No other routes. I did a quick topology diagram of the network.

It's from the host[192.168.30.10] in red area i try to connect to the switches[192.168.60.3 and 192.168.60.4] in the blue area.
I placed a computer on the .60 subnet and I was able to run rdp against it without interruption.

so the problem seems to be only with the switches. I don't understand why the firewall is dropping the traffic

Regards
December 16, 2022, 11:33:43 PM #3
Don't use floating rules unless you absolutely have to.
Put the rules where they belong, on the actual interface.
December 17, 2022, 06:42:28 AM #4
@bergstrom
Hi. yes, the switches obviously respond to requests directly - opnsense doesn't see the responses and closes the connection.
i think you can tcpdump ssh traffic and i think you will see that traffic through opnense only goes in one direction: from 30.10 client towards 60.4.
thus, the first packet  (09:26:07) with the SYN flag creates a state on the firewall, but since there are no responses, such a state (with default timeouts) lives for a little more than 30 seconds.
the packet at 09:26:42 with the PA flag is already out of state (session closed by pf). Due to the PA flag, it does not match a "AllowUserAll" rule. the nearest next rule to which it match is "Reject PrivateRanges.." (if it were not there, the "Deafult Deny" rule would work).
December 17, 2022, 02:18:22 PM #5
@Fright
Hi, you are absolutely right. I only see traffic goes in one direction in the packet capture. Thank you so much for your help!

I didnt know the switch respond to the request directly when the IP is on a different vlan. Could it be a misconfiguration of the switch?
Code Select

WS-C2960X-48TS-L#show run
Building configuration...

Current configuration : 5164 bytes
!
! Last configuration change at 18:04:32 UTC Sun Mar 8 2009 by -
! NVRAM config last updated at 18:04:27 UTC Sun Mar 8 2009 by -
!
version 15.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname WS-C2960X-48TS-L
!
boot-start-marker
boot-end-marker
!
enable secret --
!
username --
username --
aaa new-model
!
!
aaa authentication login default local
!
!
!
!
!
!
aaa session-id common
clock timezone UTC 1 0
switch 4 provision ws-c2960x-48ts-l
!
!
no ip domain-lookup
ip domain-name ad01.se
vtp mode transparent
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
vlan internal allocation policy ascending
!
vlan 20
name INTERNET
!
vlan 30
name user
!
vlan 40
name cctv
!
vlan 50
name wifi
!
vlan 60
name mgmt
!
vlan 70
name server
!
!
!
!
!
!
!
!
!
!
!
!
interface FastEthernet0
no ip address
shutdown
!
interface GigabitEthernet4/0/1
description pfsense
switchport trunk allowed vlan 1,20,30,40,50,60,70
switchport mode trunk
!
interface GigabitEthernet4/0/2
description larm
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet4/0/3
description nas
switchport access vlan 70
switchport mode access
!
interface GigabitEthernet4/0/4
switchport access vlan 20
switchport mode access
!
interface GigabitEthernet4/0/5
description laptop_tmp
switchport access vlan 50
switchport mode access
!
interface GigabitEthernet4/0/6
switchport mode access
!
interface GigabitEthernet4/0/7
switchport access vlan 50
switchport mode access
!
interface GigabitEthernet4/0/8
switchport mode access
!
interface GigabitEthernet4/0/9
switchport mode access
!
interface GigabitEthernet4/0/10
switchport mode access
!
interface GigabitEthernet4/0/11
switchport mode access
!
interface GigabitEthernet4/0/12
description vlan20
switchport access vlan 20
switchport mode access
!
interface GigabitEthernet4/0/13
description vlan30
switchport access vlan 30
switchport mode access
!
interface GigabitEthernet4/0/14
description vlan40
switchport access vlan 40
switchport mode access
!
interface GigabitEthernet4/0/15
description vlan50
switchport access vlan 50
switchport mode access
!
interface GigabitEthernet4/0/16
description vlan60
switchport access vlan 60
switchport mode access
!
interface GigabitEthernet4/0/17
switchport mode access
!
interface GigabitEthernet4/0/18
switchport mode access
!
interface GigabitEthernet4/0/19
switchport mode access
!
interface GigabitEthernet4/0/20
switchport mode access
!
interface GigabitEthernet4/0/21
switchport mode access
!
interface GigabitEthernet4/0/22
switchport mode access
!
interface GigabitEthernet4/0/23
switchport mode access
!
interface GigabitEthernet4/0/24
switchport mode access
!
interface GigabitEthernet4/0/25
switchport mode access
!
interface GigabitEthernet4/0/26
switchport mode access
!
interface GigabitEthernet4/0/27
switchport mode access
!
interface GigabitEthernet4/0/28
switchport mode access
!
interface GigabitEthernet4/0/29
switchport mode access
!
interface GigabitEthernet4/0/30
switchport mode access
!
interface GigabitEthernet4/0/31
switchport mode access
!
interface GigabitEthernet4/0/32
switchport mode access
!
interface GigabitEthernet4/0/33
switchport mode access
!
interface GigabitEthernet4/0/34
switchport mode access
!
interface GigabitEthernet4/0/35
switchport mode access
!
interface GigabitEthernet4/0/36
switchport mode access
!
interface GigabitEthernet4/0/37
switchport mode access
!
interface GigabitEthernet4/0/38
switchport mode access
!
interface GigabitEthernet4/0/39
switchport mode access
!
interface GigabitEthernet4/0/40
switchport mode access
!
interface GigabitEthernet4/0/41
switchport mode access
!
interface GigabitEthernet4/0/42
switchport mode access
!
interface GigabitEthernet4/0/43
switchport mode access
!
interface GigabitEthernet4/0/44
switchport mode access
!
interface GigabitEthernet4/0/45
switchport mode access
!
interface GigabitEthernet4/0/46
switchport mode access
!
interface GigabitEthernet4/0/47
switchport mode access
!
interface GigabitEthernet4/0/48
switchport access vlan 60
switchport mode access
!
interface GigabitEthernet4/0/49
switchport trunk allowed vlan 1,20,30,40,50,60
switchport mode trunk
!
interface GigabitEthernet4/0/50
!
interface GigabitEthernet4/0/51
!
interface GigabitEthernet4/0/52
!
interface Vlan1
ip address 192.168.20.6 255.255.255.0
!
interface Vlan30
ip address 192.168.30.4 255.255.255.0
!
interface Vlan60
description mgmt
ip address 192.168.60.4 255.255.255.0
!
ip default-gateway 192.168.60.1
ip http server
ip http authentication local
no ip http secure-server
!
ip ssh authentication-retries 2
ip ssh version 2
!
!
!
!
!
!
line con 0
exec-timeout 30 0
line vty 0 4
exec-timeout 0 0
privilege level 15
transport input ssh
line vty 5 15
exec-timeout 60 0
transport input ssh
!
ntp logging
ntp server 192.168.60.1
end


Do you have any suggestions on how to do the segmentation correctly? Or do I need to have the mgmt IP on the same subnet as the users?

December 17, 2022, 04:50:12 PM #6
looking at the config i would just probably try to connect to 192.168.30.4 instead of 192.168.60.4
December 18, 2022, 07:37:21 PM #7
Quote from: Demusman on December 16, 2022, 11:33:43 PM
Don't use floating rules unless you absolutely have to.
Put the rules where they belong, on the actual interface.

Interface groups would be the better way to go if you have multiple interfaces that need the same rules
December 18, 2022, 10:25:21 PM #8
Quote from: jclendineng on December 18, 2022, 07:37:21 PM
Quote from: Demusman on December 16, 2022, 11:33:43 PM
Don't use floating rules unless you absolutely have to.
Put the rules where they belong, on the actual interface.

Interface groups would be the better way to go if you have multiple interfaces that need the same rules

Agreed. People don't understand what floating rules are for and they use it as their main rule location.
If this guy would just put the rules where they belong his problems would go away.
Print
Go Up Pages1
User actions