[SOLVED] Default deny blocking when it shouldn't.

[RymdLord]

Hello,

I'll cut right to the chase. I have a management vlan(10) and a DMZ vlan(50).

On vlan 10 I have the following rule

             Protocol        Source       Port    Destination    Port    Gateway
 Pass   IPv4 *    vlan10net    *            *               *                   *    

 On vlan 50 I have the following rules

               Protocol                  Source              Port     Destination                Port      Gateway
Pass   IPv4 TCP/UDP        vlan50net               *            vlan50 address    53 (DNS)    *
Block        IPv4 *                vlan50net              *            vlan50 address           *            *

On vlan 50  there are 3 devices two of them are hosting web interfaces on vlan50 and I'm 100% sure both are up and working due too the 3rd device on vlan50 can access them.

This is the problem, when I try to access web interface 1 it works without any problems. But when I try to access web interface 2 it doesn't work at all. I can't even ping it. When I went to diagnose the problem I found that the "Default deny" on the vlan10 interface was blocking the connections. I have no idea what could be the problem, anyone have any ideas?


I'm running 24.1.4 and have done audit, health and security checks.
Thanks in advance!

[Seimus]

So if I understand it correctly,

You have a host in VLAN10 lets call it X.X.10.X this host tries to reach web servers/interfaces in VLAN50 lets call them X.X.50.X and X.X.50.Y.

X.X.10.X can reach X.X.50.X but cant reach X.X.50.Y?

Did you try to purge your state table?

Regards,
S.

[RymdLord]

Hello Seimus! Thank you for the reply!

I didn't try that earlier but did so now and it seems to not have worked sadly.

I really appreciate the help still!

I just checked with live logging and default block rule logging on and found this.

               Interface         Source                              Destination        Protocol           Order
Blocked     vlan10         X.X.10.X:41210                 X.X.50.Y:80          TCP                Last
Passed      vlan10         X.X.10.X:41210                 X.X.50.Y:80          TCP
Blocked     vlan10         X.X.10.X:35532                 X.X.50.Y:80          TCP
Blocked     vlan10         X.X.10.X:33658                 X.X.50.Y:80          TCP
Passed      vlan10         X.X.10.X:35532                 X.X.50.Y:80          TCP                First

It really confuses me.

Sincerely,

R

[Seimus]

Your host in VLAN10 has IP and MASK from vlan10net right?

When you check live log, do you see what are the TCP FLAGs?
Also feel free provide pictures from your OPNsense, if those are private no need to worry about hiding them

Regards,
S.

[RymdLord]

Yes the 3 devices I have tried on vlan10 all have the right IP and right mask. Yea when i check live log i see that its TCP being blocked. Could this have anyting to do with me not using the new KEA DHCP? 

Do you think reinstalling Opnsense and importing my current config could help?

Sincerely,

R

[cookiemonster]

can you provide a thorough description of the infrastructure setup i.e. virtualisation, switches, etc.? The state violation could be asymmetric routing or similar.
DHCP is not in the picture for the moment and reinstalling OPN will not solve, as the import will just put it the same way as it was i.e. same configuration.

[Seimus]

From that picture,

The Source is allowed, there are two session entries that show FW Allows them but then there is a third one that shows blocked by default rule (could be out of state). Can you click on that i Icon on that blocked rule and make a picture of it, you will see there the TCP flags.

And as well as mentioned by cookie, provide a diagram and description of your infrastructure.

Regards,
S.

[RymdLord]

Hello Everyone I reinstalled Opnsense on my hardware and it seems to have fixed it. I think the problem was that I switched hardware on the router and didn't reinstall. I essentially did a full swap of the entire system and even if it was to the same model of cpu it wasn't the same physical CPU. The lesson is reinstall Opnsense after you switch hardware.

Never mind it didn't fix it. You where right

reinstalling OPN will not solve, as the import will just put it the same way as it was i.e. same configuration.

But It fixed it for a little bit before I had to reinstall my HCI due to a password typo that locked me out. I am also 100% sure that my VM is accessible on vlan50. I checked if i could reach it with another device that I temporally connected to vlan50.

[RymdLord]

I think I found something. It might have to do with TCP-SYN/ACK. I can't see that any TCP-ACK packets being passed to 10.10.1.69 But now i also cant see the strange blocking from erlier

[RymdLord]

Hello Everyone,

First of all I want to give a huge thanks to Seimus for their help in solving my problem. As well as cookiemonster.

Together with Seimus we figured out that the problem was that the server didn't have a configured gateway on vlan50.

But the process of how we got there is the more important part.

We started by monitoring all blocked and passed traffic to figure out what was the problem. That showed that the device on vlan10 could access the server on vlan50 but that the server wasn't responding.

We did this by trying to ping the server from the device on vlan10 and then checking the firewalls state on the diagnostics tab. And looking if it passed.

After that we tried to ping the device on vlan10 from the server while logging all blocked traffic on vlan50 to check if it could access the the network. When it didn't show up on the firewall we tried pinging the gateway of vlan50 and that showed up on the firewalls logs.


That lead us to run the following commands on the server (server was running linux).
arp -a
route
They showed us that the server was missing a default gateway on its interface with vlan50.


I also want to add that there where probably two things wrong from when I first made this post. The first being to just swap the drive of OPNsense to a new system even if it had almost identical specs. And the second being forgetting to configure my server correctly.


So when you change hardware it might be a good idea to reinstall OPNsense efter the upgrade and the restoring a config file. As well as checking network configs on all devices.

Hope this helps someone!

[Seimus]

You are welcome buddy,

When Tshooting connectivity issues always remember the "Golden Trinity"
MAC table
ARP table
Route table

I will one day maybe tattoo it as well on my arm.

Regards,
S.