Messages

Hello Patrick,
Thank you for your msg.

As the "disable FW" didn't change anything, I removed this right away, knowing that disabling FW does disable NAT (As clearly mentioned in the app' menu) so I'm back to normal since my last post.

All three interface are distinctives, 1.LAN is 192.168.101.101/27; 3.LAN-WiFi is 192.168.102.101/24; 4.LAN is 192.168.103.101/28
i.e: 1.LAN can't access the NAS on 4.LAN, which is a problem for later.
i.e: 3.LAN-WiFi devices can't access 1.LAN, which is wanted.

The IPs of the devices which can't access through the FW are 192.168.101.103; 192.168.102.103; (and 192.168.102.108 as I noticed later)
All other settings are identical, worked perfectly fine before the update, the DNS are the same for all interfaces; the FW rules are copied from 1.LAN with "allow-all".

I've ticked the option to disable FW, but that didn't change any, these devices are still not able to access internet, while the others are unmoved, browsing as usual.
What's very frustrating is that some are on the same LAN, 3 (Android) are accessing the internet, 1 (laptop) is not.
And the more frustration, the less I'm able to think.
And as I know myself, if I start "trying" around, I'm going to break my OPN for sure :(

I really need your light here, suggesting debug path and steps .. 

Hello,
1.LAN RJ45 => 2 laptops
2.WAN
3.LAN RJ45 => bridge to cisco WiFi router (mostly Android devices)
4.LAN RJ45 => not tested.

Running 25.7.7, everything was good. (FW default allow all parameters)
28-Nov, Updating to 25.7.8 => 2 devices lost their Internet access (1 laptop on 1.LAN (RJ45) and 1 laptop on 3.LAN (RJ45)), while the others (Android) kept theirs.
No setup, no parameter changed during/after (compared to before, on 25.7.7)

Checking FW live view, I see these 2 laptops/IP have all TCP cnxion rejected
Since all Androids where still accessing Internet, I swap laptop1 from RJ45 on 1.LAN to WiFi on 3.LAN, same blockage; I switched laptop.2 from RJ45 on 3.LAN to WiFI on 3.LAN, same blcokage.

- How come TCP are now rejected, while everything is the same, same MAC, same static mapping IP, same rules, ...
- What should I do now ? (I tend to break things, so I prefer asking before messing around in the FW rules)

Thank you !
MSSG

I'm glad it finally worked !
And sorry for the HUNSN experience, I'll monitor mine more closely ....

The main and obvious difference I see is that all my Laptops are ThinkPads T. series , W700DS, T440p, P15-Gen2, T61p, T22, (Those not being "seen" behind OPN) while Laptop2 (The one having no problem on OPNsense) is a HP

Internet results about (ThinkPad problem OPNsense) only reports problem *installing* OPNsense ON ThinkCenters, not plugging ThinkPads behind OPN

While I understand the installation trouble (BSD, ZFS, ...) the FW itself should only be managing MAC and IPs, right ? it should not be device specific ?

Should I try a different FW (Endian, DynFI, ZeroShell) ?

Settings there seem OK. Only doubt for me is the IPv6 if disabling would help to diagnose but that's only my ignorance of IPv6.

More tests ...
The IPv6 is still OFF, and the IPv6 lines I see in the log are actually the MAC of the interfaces in an IPv6 kinda code, so it's all good, IPv6 is indeed disabled, and rules on all interfaces are now down to 18 (was 22) bevause all the IPv6 rules are out (automatic)

Laptop2 (win10), on RJ45 LAN2 .102.116, can access about everything, LAN1+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10), on WiFI LAN2, all the same
Laptop2 (win10) on RJ45 LAN1 .101.118, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10) on RJ45 LAN3 .103.119, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)

Laptop1 (Linux), on RJ45 LAN1 .101.116, can only access LAN1 interface, no WiFi AP, no NAS2
Laptop1 (Linux), on RJ45 LAN2 .102.118, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2
Laptop1 (Linux), on WiFi LAN2, all the same

Laptop4 (win10) on RJ45 LAN3 .103.116, can access LAN3 interface and NAS2 (103.118), no other LAN1+LAN2 interface, no WiFi AP (.102.102)
Laptop4 (win10) on WiFi LAN2 .102.120, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2

On the Log for each Interfaces, the only hit I see are .103.118 (NAS2) reaching out, and Laptop2 (with respectives IP depending on where it's connected) but no hit for the other two Laptops when I ping or when I access a website (i.e: this forum)

So it's probably not an OPNsense problem, and not about the OS, but it might be about the devices themselves ? (based on just one -Laptop2- being able to reach all)

Damned ... Where do I go from there ?
What setup these two may have that prevent OPNsense to see them and route them ?
Knowing that they all have access outside ...

Settings look fine.
Interfaces: Settings . What do you have there? We're looking for anything looking abnormal.
Frankly I do not know what might be the problem. It should just work by the way you have done it so it's a wild goose chase right now for what is happening and where. It might not even be in OPN.
Sure your devices' traffic should be appearing on the firewall live view and appears it doesn't.
p.s. the .255 is the broadcast address. 127 not idea. Unless the ARP table on your device is wrong for whatever reason.

Thank you for the information ! (learning every step of the way ...)
Interfaces, settings =>
(time to go to work now)

keep going, want to see the rest of the config on that page please.

Here you go !

Is it normal that Laptop1 192.168.101.116 is trying to hit 192.168.101.127 and 192.168.101.255 ?
There is nothing there, no device, etc..

If I simplify, is this correct?
Problem statement: devices on LAN1 can not get to devices on LAN3. Maybe to more but one problem at the time.
Devices on LAN1 get an ip in the range 192.168.101.0/24.

Yes sir, this is correct

The allow all rule seems to be allowing to reach the GUI of OPN but I see a lot of IPv6.
Please show the LAN1 interface setup. A screenshot please.
And a reminder. I don't know IPv6. Are you comfortable disabling it, or do you prefer someone else familiar with it to try and help?

Two posts above, I already disabled it :)
But I still see IPv6 lines in the log ?

On WAN I changed IPv6 from DHCP to None
Then same pings
No change, I still have IPv6 lines, but overall I have way less lines

Edit:
I went back to WAN to set IPv6 to DHCP, then went to each LAN to set IPv6 to None, and then back to WAN to set IPv6 to None
No change, I still have IPv6 lines in the log ?

So now time to resume diagnosing LAN1.
Test from a device on LAN1 to a device on LAN3 and watch the live logs. Filter only on interface LAN1. Rules enabled logging. What do you see?

Logging are "on" (the i-information is blue-ish)
Live log of LAN1 (attached) during all these pings
Laptop1 ping 192.168.103.101 (LAN3 interface) 100% loss
Laptop1 ping 192.68.103.118 (NAS2 on LAN3) 100% Loss
Laptop1 ping 8.8.8.8 => 0% Loss (all pass)
Laptop1 ping 178.162.131.118 (this forum) 0% loss
Same result, .. log is empty, nothing through the FW

Code Select Expand


Sophie@T440p:~
$ ping 192.168.103.101
PING 192.168.103.101 (192.168.103.101) 56(84) bytes of data.
^C
--- 192.168.103.101 ping statistics ---
11 packets transmitted, 0 received, 100% packet loss, time 10231ms

sophie@T440p:~
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=115 time=289 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=115 time=287 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=115 time=291 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=115 time=287 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 286.517/288.294/291.042/1.821 ms
sophie@T440p:~
$ ping 178.162.131.118
PING 178.162.131.118 (178.162.131.118) 56(84) bytes of data.
64 bytes from 178.162.131.118: icmp_seq=1 ttl=49 time=427 ms
64 bytes from 178.162.131.118: icmp_seq=2 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=3 ttl=49 time=440 ms
64 bytes from 178.162.131.118: icmp_seq=4 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=5 ttl=49 time=429 ms
^C
--- 178.162.131.118 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 425.294/429.298/440.282/5.641 ms
sophie@T440p:~
$


EDIT: As I changed yesterday the name of the interface (correction from IGC1 to IGC0) the filter was wrong, here is the correct one

> I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
They are logically identical but you might need to go and change the interface.
So say you have a rule on LAN1 interface. Then you clone it to use it on LAN2. You should then verify that the new rule has LAN2 as its interface that it applies to. The cloning won't know what you intended to use it for.

You can't modify the automatic ones, so you can't change the interface nor anything

Please post them here instead of assuming.

Not assuming, I did a side-by-side comparison, and did find that they are all 3 the same 23 rules, *except* for LAN1' 24th rule which is the anti-lock-out

All three attached

Second one is nice and healthy. Shows hitting the OPN LAN3 interface before going out to the device on LAN2 interface.

The first one is not healthy obviously but is not reaching the interface. You need to check the interface second. First, the network stack of that laptop. Suggest you disable and re-enable to recreate the routes when it gets a fresh dhcp lease. If set manually, verify those and the gateway you have set. I thought I had mentioned that before.

All clients are DHCP (for now) since the fresh re-install.
Network boot is always disabled on my devices (never use it, so I took a habit to remove it from the boot options)

Right then. First recap.

So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI

So that tells us your LAN2 has a firewall rule that allows to reach LAN1. One network good.

So we're back to LAN1, right ? LAN3 will be next.
So we need to view the firewall rules on this one. Compare to the known good. Share it here as before (lost track if is the one you shared already). And share your interface setup. Could it be set as not /24  I wonder.

I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
Based on what ? what is the parameters that leads to such decision ?
AFAIK, all three LANs are all the same, all with the 20-ish automatic rules and the two initial extras (to avoid "lambda :-p ) ones
I will try a side by side comparison then ...
All the exact same 23 rules, except LAN1 has a 24th self anti-lockout

For the purpose of diagnosing the network, leave IoT, mobiles phones, smart toasters, etc. out of the picture. If you have a device with a console like a laptop that you seem to have, just stick with them. Keep it simple.
Once problem solved, then you can start looking at those in the knowledge your firewall and rules are not in the way.

Sure thing !

p.s. And on your iperf test. a) you did set the server in listening mode (-s), right? ;b) the error suggests there was no return so could be the missing rule on that interface to get the packets back to sender.
In fact, time to do it in pairs.

iperf was set as shown in the code: -c (client) then IP, then -d (double / return)
The -s option is for serveur, taht's why I didn't use it, will try that

Code Select Expand


sophie@T440p:
$ iperf -s 192.168.103.101 -d
WARNING: option -d is not valid for server mode
iperf: ignoring extra argument -- 192.168.103.101
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------


Been in that state for about 5min now ...

Traceroute from Laptop1 to NAS2:

Code Select Expand


sophie@T440p:~
$ traceroute 192.168.103.118
traceroute to 192.168.103.118 (192.168.103.118), 30 hops max, 60 byte packets
1  * * *
2  * * *
3  * * *
4  * * *
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *


Tracert from Laptop2 to NAS2

Code Select Expand


C:\Users\xxxx>tracert 192.168.103.118

tracing route to NAS653D [192.168.103.118]
over a maximum of 30 hops:
1  <1 ms  <1 ms  <1 ms 192.168.102.101
2  <1 ms  <1 ms  <1 ms NAS653D [192.168.103.118]

Trace complete.