Messages

I tested this scenario with Pfsense as well and got the same results. I have not tested other firewalls.

Well, that's rather funny don't you think, looking at your initial statement   ;)

But two firewalls behaving in the same way kind of support the idea that this is normal.

That's because the underlaying issue is still there (it's _not_ normal), did you test with hardware offloading disabled as mentioned in my last post ?

===============
Yeap, it is funny :). I originally thought the auto generated rules were the cause, but since they are not, its not relevant anymore.

As for testing with the hardware offloading disabled, I think this is disabled by default, so any tests carried out had this disabled by default.

Also, if this is not normal and confirmed by more than one user, isnt this a priority issue?

I am now back again sitting on the fence on whether we should be concerned about this or not?

Thank you everyone for your input and support.

I tested this scenario with Pfsense as well and got the same results. I have not tested other firewalls. But two firewalls behaving in the same way kind of support the idea that this is normal.

In the end i like what I am seeing on Opnsense, so I am sticking with it even through the title says othwerwise :)

[Houston we have a problem"]

As they say in the famous movie "Houston we have a problem"

I kept drilling down as I kept getting replies to my pings.

In the end to remove any middleman issues and confirm once and for all this is opnsense issue, I removed the switch, the virtualization and went baremetal.

Testing env:
Two baremetal win11 PCs.
4 port fresly installed Opnsense.

PCs: Both baremetal:
PC1 - connected directly to LAN port on the opnsense firewall = IP add 10.1.1.1
PC2 - connected directly to OPT1 port on the opnsense firewall = IP add 10.2.2.2

Opnsense:
Fresh install, the only config I added was to enable OPT1 and assing IP Address 10.2.2.254. Nothing else changed.

Test result, I still get a response to the ping.
Steps:
Issued the ping command form PC2, at first dont get a response
However, as soon as you issue the ping command on PC1, both PCs can ping each other.

Please see the screenshots.
Note: The system due to the size does not allow me to attach all the screenshots in one go. I will add them one by one.

Isn't this just ICMP hole punching?  https://en.wikipedia.org/wiki/ICMP_hole_punching

The ping was correctly blocked until you opened up the return by starting from the other way.  It's the whole way a lot of zero config apps work.

Thank you for the input. Although I understand and agree in a sense on what you saying, although not 100% sure I know the answer to that.

If someone with definite knowledge can confirm its not a security issue I am happy with that although at the beginning I thought it was the auto generated rules was the cause.

Also in the link you shared it shows that an attacker can pretend to be replying to your ping request in return causing the remote PC to respond and we have an open connection ready to be exploited. But i guess thats a conversation for another day.

I guess the question now is is this a security issue or is it expected as you and
Monviech mentioned.

Image addition:

Image addition:

Image addition:

Image addition:

Image addition:

[Houston we have a problem"]

Whats is your take on this please?
Please see the attached screenshot.

I explained my take extensivily in previous post:

- Go back to the drawing board
- Simplify your setup
- From here proof your initial statement that "Automatically generated rules" allows traffic that isn't expected to be allowed (ICMP or whatever.)

So a default install, 1 WAN, 1 LAN and 1 OPT interface, no VLAN's, no manual config except for 3 interface IP configs. If in this setup you can proof your statement many people are willing to look into your issue. Should take you less time than posting screenshoits of state reset buttons.

For me, I'm too old to look into virtualised infra's with VLAN trunk ports without a detailed low level design and not knowing if basic network skills are in place.

As they say in the famous movie "Houston we have a problem"

I kept drilling down as I kept getting replies to my pings.

In the end to remove any middleman issues and confirm once and for all this is opnsense issue, I removed the switch, the virtualization and went baremetal.

Testing env:
Two baremetal win11 PCs.
4 port fresly installed Opnsense.

PCs: Both baremetal:
PC1 - connected directly to LAN port on the opnsense firewall = IP add 10.1.1.1
PC2 - connected directly to OPT1 port on the opnsense firewall = IP add 10.2.2.2

Opnsense:
Fresh install, the only config I added was to enable OPT1 and assing IP Address 10.2.2.254. Nothing else changed.

Test result, I still get a response to the ping.
Steps:
Issued the ping command form PC2, at first dont get a response
However, as soon as you issue the ping command on PC1, both PCs can ping each other.

Please see the screenshots.
Note: The system due to the size does not allow me to attach all the screenshots in one go. I will add them one by one.

Are you still using LAN as the parent of all of the VLANs?

Yes.

However, I come across this question couple of times now. Is that not how we supposed to do it?

The default LAN rule allows all.  I know there is a recommendation not to mix tagged and untagged traffic on the same interface but I forget the exact details for why.  If you change your VLANs to use something other than LAN as the parent, you may see things work as you expect.

That all said, why not try testing with separate interfaces and no VLANs.  That way you eliminate all of the variables of incorrect VLAN setup, etc.  It will help isolate if the problem is truly OPNSense or not.

Ahh now I understand. I will dig out some 4 ports NIC cards. Ok for this i need to swap from 2 port NIC to 4 port NIC so i can use the 3rd port as VLANs parent interface.

So the new setup will look like this:

Port 1 = WAN
Port 2 = LAN
Port 3 = Parent Interface for all VLANS. - This needs to be Trunk/Tagged Ports for all VLANS?

TIA

Are you still using LAN as the parent of all of the VLANs?

Yes.

However, I come across this question couple of times now. Is that not how we supposed to do it?

[To remove any possibility that this is vmware issue I used two win11 baremetal PCs for testing and also moved the opnsense to baremetal pc.]

Below is what I tested so far and to be honest I am concerned with the results.

I realise some users spent some valuable time on this and I appriciate it, however i think for the benefit of everyone who is using opnsense I believe we need to get to the bottom of this.

If my tests are not flowed somehow, and i dont see how thats posisble as its simple ping test, when all baremetals now, than we are looking at a bug or a bigger issue.

I know some will dismiss this without even looking into it, but I am concerned. Something is not right with opnsense.

To remove any possibility that this is vmware issue I used two win11 baremetal PCs for testing and also moved the opnsense to baremetal pc.

The current test setup:
PC-S1 - Source PC - win 11   (baremetal) -  VLAN140
PC-S2 - Source PC - Ubuntu  (VM) -             VLAN210
PC-D1 - Destination PC win 11(baremetal) -  VLAN190
Opnsense (baremetal)

Test results:

VLAN140 disabled all the rules. (NO rules on floating or on vlan140 - all disabled)
Test results -
Started pinging from - PC-S1 - Ping did NOT work -however as soon as i started pinging back form PC-D1 both source and destination can ping each other.

Than i though maybe this is a windows issue, so i tested with an ubuntu (vm) and the same results.

following that, i thought maybe because there are some config on VLAN140 causing the issue even though its disabled. So moved the test to VLAN210 which has no config at all. And again the resutls are the same.

Some of you requested more basic testing, but i already (I know not the brightest move) switched to opnsense baremetal thinking this is happening due to the FW virtual issue and i wont get the same problem on baremetal, so its not easy to remove all the config again as the household will probably not be happy. Therefore i did the testing while also trying to keep the home internet working.

[Yes.]

Did you configure a vswitch, and added a port group with the VLAN ID of 4095? And connected this Portgroup with 4095 as E1000 or vmxnet3 nic to the Opnsense so its a trunk port (accepting all vlan tags)?
And then created port groups on that vswitch with all additional vlan IDs 100-190... you have and connected those to your VMs?

I only configure Opnsenses on ESXi with PCIe passthrough when they need VLANs, I tested it and they don't have problems with states as described above. I can't replicate it there. I also run a few opnsenses that have one vnic per portgroup for different networks. It also doesn't happen there.

So, if anybody has a setup that uses portgroups with VLANs...

Did you configure a vswitch, and added a port group with the VLAN ID of 4095?
Yes.
And connected this Portgroup with 4095 as E1000 or vmxnet3 nic to the Opnsense so its a trunk port (accepting all vlan tags)?
Yes.
And then created port groups on that vswitch with all additional vlan IDs 100-190... you have and connected those to your VMs?
Yes.

=======================
However, as some people raised concern about the setup being virtual and possibly causing the issue I dug up a  mini PC did fresh baremetal opnsense install and run the tests, to my suprise i got the same results.

So virtual or baremetal does not make a difference, as soon as you start pinging from vlan190 to vlan160, vlan160 can start to ping vlan190 back.

Just so it does not get lost in the conversation, vlan160 only gets response to pings if i start pinging from vlan190. If i start the ping on vlan160 the pings does not work, It only get response when i start the ping from vlan190.

Therefore anyone wants to test this scenario suggest you clear the firewall states or better reboot if possible and start pinging from the vlan that has NO config and watch the result give it 10-15 seconds and only than start pinging from the VLAN has the permit statement.

as for your tests, i am suprised in a sense that you did not get the same results. In your setup do both vlans have rules on them configured? perhaps this behaviour only happens when there is no config on one of the vlans?

Did you create the VLANs in OPNsense? It looks like it. How is the parent interface passed into the VM? If this is all virtual, the common approach is to define all VLANs on ESXi and pass a matching number of virtual interfaces into the firewall VM. Possibly alle your VLANs are not really VLANs at all?

How is the parent interface passed into the VM?
I use esxi and its virtual interface, not passed through.

If this is all virtual, the common approach is to define all VLANs on ESXi and pass a matching number of virtual interfaces into the firewall VM.
I think thats would not be posisble as ESXi have limit of 10 NICs max per VM. so if you want to have more than 10 vlans which i do that approach wont be posssible.

Possibly alle your VLANs are not really VLANs at all?
The same Infrastructure works ok with pfsense, but pfsense has a bug i hate hence the adventure to opnsense :)

Therefore althought its always a possiility due to misconfiguration i think its unlikely?

Only one statement CAN be true: You discovered a major security bug/flaw in PF and/or OPNSense OR you have issues in your virtual environment, network topology, firewall config etc.

Because your issue is quite simple, allowing ICMP echo request / reply between two interfaces, I suggest you create a more simple setup to debug and report on with a basic and easy to understand topology and ruleset. Your virtual so deploying an extra VM should be an easy task for such a potential high impact security bug/flaw (not only impacting OPNSense).

So a default setup with LAN + WAN and after that creating an extra OPT interface to introduce a second LAN segment. Keep EVERYTHING default just configure the interface IP's and DO NOTHING else. To make it super transparent, don't mess with VLAN interfaces YET. If you need just create static mappings between your OPNSense interfaces and your existing VLANs by creating VLAN port groups in vSphere VDS or Open vSwitch on KVM or whatever you are using.

Now if you can confirm that in this DEFAULT setup you can ping from a host in the OPT network to a host in the LAN network it's time to get scared. The other way around, ping from _DEFAULT_ LAN network to OPT network, will work as explained in previous posts.

I couldn't bother to plow through your ruleset (too much noise), but I did check your statement at a system here with ten's of raw interfaces, bridges and VLANs (both LAGG and non-LAGG) and couldn't reproduce it on any.

From my perspective a misconfiguration is VERY likely, but I'm eager to hear about the issues you find in the above "test" setup.

I agree it would be good to get to the bottom of this.

Really busy day at work so have not had the chance to do the test you asked yet. But I did some thinkering and think I figured it out. I need another pair of eyes to confirm if this is the case.

VLAN190 has permit all rule
VLAN160 has NO rule

If i start the ping from vlan190 to vlan160 (vlan190 has permit all statement) it works ok (as expected).

If i reboot both PCs and the firewall so there is no established connection as far as the firewall is concerned and try to ping from vlan160 (which does not have any firewall statements), it fails (as expected).

So far the firewall does what its expected.

However, the point it gets unexpected is as follows:

After the reboot all connections states is cleared. so if i start the ping from vlan160 to vlan190, it fails as expected.
However, I noticed the second i start pinging from vlan190 back to vlan160 naturally vlan190 works ok, but because the firewall now has established an open state between this two IP Addresses it allows vlan160 to ping back vlan190.

if i stop the ping from vlan190 and reset the state table the ping stops which supports my findings?

Whats is your take on this please?
Please see the attached screenshot.