Messages

The main and obvious difference I see is that all my Laptops are ThinkPads T. series , W700DS, T440p, P15-Gen2, T61p, T22, (Those not being "seen" behind OPN) while Laptop2 (The one having no problem on OPNsense) is a HP

Internet results about (ThinkPad problem OPNsense) only reports problem *installing* OPNsense ON ThinkCenters, not plugging ThinkPads behind OPN

While I understand the installation trouble (BSD, ZFS, ...) the FW itself should only be managing MAC and IPs, right ? it should not be device specific ?

Should I try a different FW (Endian, DynFI, ZeroShell) ?

Settings there seem OK. Only doubt for me is the IPv6 if disabling would help to diagnose but that's only my ignorance of IPv6.

More tests ...
The IPv6 is still OFF, and the IPv6 lines I see in the log are actually the MAC of the interfaces in an IPv6 kinda code, so it's all good, IPv6 is indeed disabled, and rules on all interfaces are now down to 18 (was 22) bevause all the IPv6 rules are out (automatic)

Laptop2 (win10), on RJ45 LAN2 .102.116, can access about everything, LAN1+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10), on WiFI LAN2, all the same
Laptop2 (win10) on RJ45 LAN1 .101.118, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10) on RJ45 LAN3 .103.119, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)

Laptop1 (Linux), on RJ45 LAN1 .101.116, can only access LAN1 interface, no WiFi AP, no NAS2
Laptop1 (Linux), on RJ45 LAN2 .102.118, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2
Laptop1 (Linux), on WiFi LAN2, all the same

Laptop4 (win10) on RJ45 LAN3 .103.116, can access LAN3 interface and NAS2 (103.118), no other LAN1+LAN2 interface, no WiFi AP (.102.102)
Laptop4 (win10) on WiFi LAN2 .102.120, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2

On the Log for each Interfaces, the only hit I see are .103.118 (NAS2) reaching out, and Laptop2 (with respectives IP depending on where it's connected) but no hit for the other two Laptops when I ping or when I access a website (i.e: this forum)

So it's probably not an OPNsense problem, and not about the OS, but it might be about the devices themselves ? (based on just one -Laptop2- being able to reach all)

Damned ... Where do I go from there ?
What setup these two may have that prevent OPNsense to see them and route them ?
Knowing that they all have access outside ...

Settings look fine.
Interfaces: Settings . What do you have there? We're looking for anything looking abnormal.
Frankly I do not know what might be the problem. It should just work by the way you have done it so it's a wild goose chase right now for what is happening and where. It might not even be in OPN.
Sure your devices' traffic should be appearing on the firewall live view and appears it doesn't.
p.s. the .255 is the broadcast address. 127 not idea. Unless the ARP table on your device is wrong for whatever reason.

Thank you for the information ! (learning every step of the way ...)
Interfaces, settings =>
(time to go to work now)

keep going, want to see the rest of the config on that page please.

Here you go !

Is it normal that Laptop1 192.168.101.116 is trying to hit 192.168.101.127 and 192.168.101.255 ?
There is nothing there, no device, etc..

If I simplify, is this correct?
Problem statement: devices on LAN1 can not get to devices on LAN3. Maybe to more but one problem at the time.
Devices on LAN1 get an ip in the range 192.168.101.0/24.

Yes sir, this is correct

The allow all rule seems to be allowing to reach the GUI of OPN but I see a lot of IPv6.
Please show the LAN1 interface setup. A screenshot please.
And a reminder. I don't know IPv6. Are you comfortable disabling it, or do you prefer someone else familiar with it to try and help?

Two posts above, I already disabled it :)
But I still see IPv6 lines in the log ?

On WAN I changed IPv6 from DHCP to None
Then same pings
No change, I still have IPv6 lines, but overall I have way less lines

Edit:
I went back to WAN to set IPv6 to DHCP, then went to each LAN to set IPv6 to None, and then back to WAN to set IPv6 to None
No change, I still have IPv6 lines in the log ?

So now time to resume diagnosing LAN1.
Test from a device on LAN1 to a device on LAN3 and watch the live logs. Filter only on interface LAN1. Rules enabled logging. What do you see?

Logging are "on" (the i-information is blue-ish)
Live log of LAN1 (attached) during all these pings
Laptop1 ping 192.168.103.101 (LAN3 interface) 100% loss
Laptop1 ping 192.68.103.118 (NAS2 on LAN3) 100% Loss
Laptop1 ping 8.8.8.8 => 0% Loss (all pass)
Laptop1 ping 178.162.131.118 (this forum) 0% loss
Same result, .. log is empty, nothing through the FW

Code Select Expand


Sophie@T440p:~
$ ping 192.168.103.101
PING 192.168.103.101 (192.168.103.101) 56(84) bytes of data.
^C
--- 192.168.103.101 ping statistics ---
11 packets transmitted, 0 received, 100% packet loss, time 10231ms

sophie@T440p:~
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=115 time=289 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=115 time=287 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=115 time=291 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=115 time=287 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 286.517/288.294/291.042/1.821 ms
sophie@T440p:~
$ ping 178.162.131.118
PING 178.162.131.118 (178.162.131.118) 56(84) bytes of data.
64 bytes from 178.162.131.118: icmp_seq=1 ttl=49 time=427 ms
64 bytes from 178.162.131.118: icmp_seq=2 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=3 ttl=49 time=440 ms
64 bytes from 178.162.131.118: icmp_seq=4 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=5 ttl=49 time=429 ms
^C
--- 178.162.131.118 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 425.294/429.298/440.282/5.641 ms
sophie@T440p:~
$


EDIT: As I changed yesterday the name of the interface (correction from IGC1 to IGC0) the filter was wrong, here is the correct one

> I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
They are logically identical but you might need to go and change the interface.
So say you have a rule on LAN1 interface. Then you clone it to use it on LAN2. You should then verify that the new rule has LAN2 as its interface that it applies to. The cloning won't know what you intended to use it for.

You can't modify the automatic ones, so you can't change the interface nor anything

Please post them here instead of assuming.

Not assuming, I did a side-by-side comparison, and did find that they are all 3 the same 23 rules, *except* for LAN1' 24th rule which is the anti-lock-out

All three attached

Second one is nice and healthy. Shows hitting the OPN LAN3 interface before going out to the device on LAN2 interface.

The first one is not healthy obviously but is not reaching the interface. You need to check the interface second. First, the network stack of that laptop. Suggest you disable and re-enable to recreate the routes when it gets a fresh dhcp lease. If set manually, verify those and the gateway you have set. I thought I had mentioned that before.

All clients are DHCP (for now) since the fresh re-install.
Network boot is always disabled on my devices (never use it, so I took a habit to remove it from the boot options)

Right then. First recap.

So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI

So that tells us your LAN2 has a firewall rule that allows to reach LAN1. One network good.

So we're back to LAN1, right ? LAN3 will be next.
So we need to view the firewall rules on this one. Compare to the known good. Share it here as before (lost track if is the one you shared already). And share your interface setup. Could it be set as not /24  I wonder.

I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
Based on what ? what is the parameters that leads to such decision ?
AFAIK, all three LANs are all the same, all with the 20-ish automatic rules and the two initial extras (to avoid "lambda :-p ) ones
I will try a side by side comparison then ...
All the exact same 23 rules, except LAN1 has a 24th self anti-lockout

For the purpose of diagnosing the network, leave IoT, mobiles phones, smart toasters, etc. out of the picture. If you have a device with a console like a laptop that you seem to have, just stick with them. Keep it simple.
Once problem solved, then you can start looking at those in the knowledge your firewall and rules are not in the way.

Sure thing !

p.s. And on your iperf test. a) you did set the server in listening mode (-s), right? ;b) the error suggests there was no return so could be the missing rule on that interface to get the packets back to sender.
In fact, time to do it in pairs.

iperf was set as shown in the code: -c (client) then IP, then -d (double / return)
The -s option is for serveur, taht's why I didn't use it, will try that

Code Select Expand


sophie@T440p:
$ iperf -s 192.168.103.101 -d
WARNING: option -d is not valid for server mode
iperf: ignoring extra argument -- 192.168.103.101
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------


Been in that state for about 5min now ...

Traceroute from Laptop1 to NAS2:

Code Select Expand


sophie@T440p:~
$ traceroute 192.168.103.118
traceroute to 192.168.103.118 (192.168.103.118), 30 hops max, 60 byte packets
1  * * *
2  * * *
3  * * *
4  * * *
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *


Tracert from Laptop2 to NAS2

Code Select Expand


C:\Users\xxxx>tracert 192.168.103.118

tracing route to NAS653D [192.168.103.118]
over a maximum of 30 hops:
1  <1 ms  <1 ms  <1 ms 192.168.102.101
2  <1 ms  <1 ms  <1 ms NAS653D [192.168.103.118]

Trace complete.


quick check before reading your last post properly. Is the problem to reach only one device on the other network or any and all (that you can tell of course) ?

Good question (to rule out the network but rather incriminate one host)

So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI

while Laptop4, even though also win10 (on LAN3) can't ping other LANs, I therefore assume it won't reach any devices on these
Ping 192.168.101.101, 100% loss
Ping 192.68.101.116 (Laptop1) 100% loss
Ping 192.168.102.101 100% loss
Ping 192.168.102.116 loss

While the Linux and android devices can't
Laptop1 (LAN1) can't ping Laptop4; Laptop3 can't ping NAS2; can't ping Interfaces (other than its own)
Laptop3 (LAN3) can't ping Laptop1; Laptop4; can ping NAS2 (but can't reach its GUI); can't ping Interfaces (other than its own)
Android devices can't reach anything internal (on any LAN, including their own) neither can they browse to the WiFI AP GUI they are attached to
Laptop4 when booted to Qubes-os neither, but that OS is so secured, so far I didn't even mentioned it for networking tests

Then is not getting to OPN.
Worth remembering what is meant to happen:
Devices on the same netwkork i.e. on LAN1 will NOT go _through_ OPN. So all devices on that switch will only go through the switch, port to port. Unless you are enabling some routing funtions on that switch. If it is unmanaged, then is a moot point.

Devices on different networks i.e. on different interfaces of OPN will go from one to the other _through_ OPN. So a packet from say client A on LAN1 192.168.101.116 to LAN3 on 192.168.103.101 will go _through_ OPN and once all is logged in OPN, will appear there.

If it doesn't, then is not hitting OPN.
At that point you must go back to checking physical connections, interface setup on OPN and your switches.

Start breaking up the problem with this overview and go in a systematic manner.
Another suggestion: skip pings for a moment and try other type of connections. iperf, maybe a webserver, etc. Easy with a laptop on each end.

Thank you
All connection are good (RJ45 IN, led ON, etc..)
All devices on each Interfaces are reaching out to the Internet through OPN
All Interface set to the same as LAN1 (naming, rules, ..)
DNS is autom
All switches are unmanaged, and WiFI AP router is set to bridge, unmanaged

Worth noting: I have access to NAS2 (LAN3) GUI from Laptop2 (LAN2) !!
This is my work laptop, and the only one device out of all which I don't need/want to access any NAS (or anything else on my network for that matter) ... aaarrgggg wwhhyyyy ?

iperf:

Code Select Expand

sophie@T440p:~
$ iperf -d 192.168.103.101
WARNING: option -d is not valid for server mode
iperf: ignoring extra argument -- 192.168.103.101
Usage: iperf [-s|-c host] [options]
Try `iperf --help' for more information.
sophie@T440p:~
$ iperf -c 192.168.103.101 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.103.101, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
ERROR: expected reverse connect did not occur
tcp connect failed: Connection timed out
[  1] local 0.0.0.0 port 0 connected with 192.168.103.101 port 5001
sophie@T440p:~
$


just filter on LAN1 please, and ping from the laptop on that network.
If it is not appearing on the live log, the packet is not hitting the firewall. That simple.

Filter to LAN1 only
Laptop1 to ping LAN3 interface,
No line appearing in the log ... (waited a couple of minutes with ping still on)

From the laptop, ping LAN3 interface 192.168.103.101
I don't see any such appearing in the log

But I see 192.168.101.116 (Laptop1) trying to hit 192.168.101.127 which is an IP leading to none ... strange to me, but maybe there is a rational ?