Hello,
Today, I'm going to (try) connect my NAS (QNAP with double RJ45) to LAN1
LAN1: 192.168.101.101/24 DHCP 192.168.101.102-122
Unmanned switch with 2x NAS; 1x Laptop1; 1x Laptop2
Turn-on NAS: LEASE 192.168.101.104 and 192.168.101.105 (Which are already attributed in Static to other devices, OPNsense should not re-attribute these !)
Change these two dynamic IPs to Statics:
Static DHCPv4 for NAS 1st RJ45 = 192.168.101.111
Static DHCPv4 for NAS 1st RJ45 = 192.168.101.112
Can't connect to NAS GUI, can't even ping these IPs
And I have something weird on the dashboard:
2024-10-04T06:26:24-04:00
<6>arp: 192.168.101.112 moved from 24:5e:be:5c:86:6c to 24:5e:be:5c:87:6d on igc0
2024-10-04T06:26:22-04:00
<6>arp: 192.168.101.112 moved from 24:5e:be:5c:86:6d to 24:5e:be:5c:86:6c on igc0
Why is it swapping MAC every 2-3 seconds on this IP to an unknown MAC and back to known MAC ?
I've created a third STATIC Lease to the "new" MAC with 192.168.101.113 and now it stopped swapping, but I still can't access the GUI
What am I doing wrong ? it should be straightforward, plug, wait for IP, and connect, no ?
Static leases must lie outside of your dynamic range.
Quote from: Patrick M. Hausen on October 04, 2024, 12:57:31 PM
Static leases must lie outside of your dynamic range.
TY for your answer, but ...
I have a DHCP for about 20 addresses per LAN, out of which I freeze (static) some, therefore inside the same lease span
Every other devices have been frozen to static the same way, including those in LAN2 (through the WiFi router AP and limited to known devices only) why would it be different for the NAS ?
I'm not sure about your statement ?
Nothing fundamentally different. What you did for the other devices can lead to the same problem. It's a question of timing. You were lucky until now.
Just don't do it, it's not a supported configuration. When I wrote "must" I meant MUST like in RFCs ;)
Quote from: Patrick M. Hausen on October 04, 2024, 02:49:30 PM
Nothing fundamentally different. What you did for the other devices can lead to the same problem. It's a question of timing. You were lucky until now.
Just don't do it, it's not a supported configuration. When I wrote "must" I meant MUST like in RFCs ;)
TY, but there again it's misleading (or confusing) if we are not supposed to use the dynamic-to-static why does the GUI offers a very simple "+" button to actually do just that ?
And for each interface, the simple option to add a frozen IP/MAC in the "DHCP Static Mappings for this interface."
Then, what should I have used to freeze all of these addresses to static if not the dhcp static one ? (neighbour ?)
I've hard reset the NAS, now I can PING it (from OPNsense and from Laptop1) on its static address 192.168.101.112 but still can't access its GUI
Do I need to open something in the FW rules (I'm scared to change anything in there)
Quote from: MarieSophieSG on October 04, 2024, 03:22:31 PM
TY, but there again it's misleading (or confusing) if we are not supposed to use the dynamic-to-static why does the GUI offers a very simple "+" button to actually do just that?
To conveniently copy the MAC address to the static reservation form? Then you are supposed to enter an IP address that is of course from the same network but outside the dynamic range you configured in the DHCP settings for that interface.
Think of each static reservation as another "range" covering one host. Ranges must not overlap.
Quote from: MarieSophieSG on October 04, 2024, 03:24:31 PM
I've hard reset the NAS, now I can PING it (from OPNsense and from Laptop1) on its static address 192.168.101.112 but still can't access its GUI
Do I need to open something in the FW rules (I'm scared to change anything in there)
If you have the "allow all" rules still in place, there's nothing you could change.
Watch what happens when you try to access the UI:
Firewall > Log Files > Live View
Interfaces > Diagnostics > Packet Capture
Quote from: Patrick M. Hausen on October 04, 2024, 03:34:07 PM
Quote from: MarieSophieSG on October 04, 2024, 03:22:31 PM
TY, but there again it's misleading (or confusing) if we are not supposed to use the dynamic-to-static why does the GUI offers a very simple "+" button to actually do just that?
To conveniently copy the MAC address to the static reservation form? Then you are supposed to enter an IP address that is of course from the same network but outside the dynamic range you configured in the DHCP settings for that interface.
Think of each static reservation as another "range" covering one host. Ranges must not overlap.
Hum ... I see ..
So I should have (same for the other 2) :
LAN1 192.168.101.101/24 (existing)
DHCPv4 192.168.101.102-122 (existing) moved to 192.168.101.115-122 (to be)
And keep my current static 192.168.101.102; 192.168.101.103, ... 192.168.101.111, 192.168.101.112
Does that looks better ?
Quote from: Patrick M. Hausen on October 04, 2024, 03:37:24 PM
If you have the "allow all" rules still in place, there's nothing you could change.
Watch what happens when you try to access the UI:
Firewall > Log Files > Live View
Interface Time Source Destination Proto Label
IGC0_SWITCH1_ETH1_CAT7green 2024-10-04T10:35:51-04:00 RS39:67 192.168.101.112:68 udp allow access to DHCP server
IGC0_SWITCH1_ETH1_CAT7green 2024-10-04T10:35:51-04:00 192.168.101.112:68 255.255.255.255:67 udp allow access to DHCP server
Quote from: Patrick M. Hausen on October 04, 2024, 03:37:24 PM
Interfaces > Diagnostics > Packet Capture
Set capture to interface LAN1 and IP 192.168.101.112
View capture
Interface Timestamp output
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:08.012951 IPv4, length 98: 192.168.101.101 > 192.168.101.112: ICMP echo request, id 64245, seq 6775, length 64
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:08.013069 IPv4, length 98: 192.168.101.112 > 192.168.101.101: ICMP echo reply, id 64245, seq 6775, length 64
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:09.023283 IPv4, length 98: 192.168.101.101 > 192.168.101.112: ICMP echo request, id 64245, seq 6776, length 64
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:09.023412 IPv4, length 98: 192.168.101.112 > 192.168.101.101: ICMP echo reply, id 64245, seq 6776, length 64
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:09.574796 IPv4, length 146: 192.168.101.112.6881 > 31.10.77.218.14994: UDP, length 104
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:09.718070 IPv4, length 359: 31.10.77.218.14994 > 192.168.101.112.6881: UDP, length 317
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:10.031770 IPv4, length 98: 192.168.101.101 > 192.168.101.112: ICMP echo request, id 64245, seq 6777, length 64
IGC0_SWITCH1_ETH1_CAT7green
igc0 2024-10-04
10:51:10.031897 IPv4, length 98: 192.168.101.112 > 192.168.101.101: ICMP echo reply, id 64245, seq 6777, length 64
From my innexistent knowledge, this looks good, I should be able to access it no problem, right ?
Done,
New network setup:
- ETH1 LAN1 192.168.101.101/24
-- Statics 192.168.101-115;
-- DHSC 192.168.101.116-122
- ETH3 LAN2 192.168.102.101/24
-- Statics 192.168.102-115;
-- DHSC 192.168.102.116-122
- ETH4 LAN3 192.168.103.101/24
-- Statics 192.168.103-115;
-- DHSC 192.168.103.116-122
Ping 192.168.101.111
From Laptop1 (LAN1) = 100% loss
From OPNsense = 100% loss Host is down
Ping 192.168.101.112
From Laptop1 (LAN1) = 0% loss
From OPNsense = 0% loss
But still no access to GUI :(
FW log same as earlier:
Interface Time Source Destination Proto Label
IGC0_SWITCH1_ETH1_CAT7green 2024-10-04T16:19:08-04:00 192.168.101.101 192.168.101.112 icmp let out anything from firewall host itself
IGC0_SWITCH1_ETH1_CAT7green 2024-10-04T16:17:50-04:00 192.168.101.101 192.168.101.112 icmp let out anything from firewall host itself
Weirdly enought (at least to me) I see traffic going in/out of 192.168.101.111 and 192.168.101.112 in "reporting / Traffic"
I've nmap 192.168.1.1/16
Found out that the NAS kept (despite the reset) its previous IPs 192.168.211.211 and 192.168.211.212
I've set a VLAN up with 192.168.211.210/24 and DHCP 192.168.211.211 192.168.211.212
added the interface, set the NAT up, set the FW rule up (allow all)
Can't even ping 192.168.211 -212
VLAN on LAN1 Not working so well wirh ummanaged switch
VLAN0.211 192.168.211.210/24
DHCP 192.168.211.211-222
FW rule "allow all"
No ping, no connection, so ...
disable LAN2, disable LAN3 (otherwise they would have their DHCP inside the LAN1 DHCP)
Changed LAN1 from 192.168.101.101/24 to 192.168.101.101/16
Added LAN1 DHCPv4 range 192.168.211.210-222
FW rules unchanged (LAN1 Laptop1 is still connected to Internet)
Ping 192.168.211.211 nothing 192.168.211.212 nothing
Disable IDS+IPS ... now I can ping 192.168.211.212, but still can't access the GUI
ERR_CONNECTION_REFUSED
Created Virtual IPs 192.168.211.210/24 but here again, no chance
Unless someone has an idea, next step is to hard-reset again (but fro 30sec) the NAS and re-do its entire config from scratch :/
Even weirder ....
I gave up, but before doing the hard reset I tried one last thing:
Unplugged the NAS and renoved it from the rack, then back to the workbench (LAN3), HDMI screen + Keyboard and direct sell access
On that work bench is Laptop3, currently running Win10
Then NAS got an IP 192.168.103.116 (the first of the DHCP) and I could access its web GUI from that Laptop
NAS can't access internet, no matter how I configure it, with manual DNS, with Automatic DNS; with manual IP, with DHCP IP
It just won't connect outside of the internal network
But at least I accessed the GUI !
Then, once I made sure everything was autom/DHCP, I plug it back to on its shelf, on LAN1, and here, I again can't access it from Laptop1 ... I really don't understand, as both LAN1 and LAN3 have cloned FW rules and are set to not block private IP
Do I need to give it special access or rediect ports or something ?
aaarrrggg found it !
Can't access in HTTP, it must be HTTPS !
And can't access with simple IP, I need to add the port
And I've disabled IDS/IPS
NO = HTTP://192.168.101.116
NO = HTTPS://192.168.101.116
YES = HTTPS://192.168.101.116:321
OK, no I have access to the GUI from Laptop1 (LAN1), login+TFA, fine !
Even tough "Services: ISC DHCPv4: Leases" shows status: "offline"
And I've lost connection to GUI again ...
Reset (3sec) again, I can now ping both IPs from the NAS, but still can't access the GUI :(
Https://192.168.101.111 => Ping ok, no GUI
Https://192.168.101.112 => Ping ok, no GUI
Bingo ! of course I need the port ...
Https://192.168.101.112:321 => Ping ok, GUI ok
Https://192.168.101.112:321 => Ping ok, GUI ok
Now that the two interface are no longer Port-Trunking, I have full access in/out
Since the NAS has 2x 2,5GbE and so does the swicth and so does the RS39 OPNsense box, I guess that won't change much.
ok, that's a very good achievement
Now trying:
Map local folder
Access NAS from LAptop2 (LAN2) and Laptop3&4 (LAN3) as for now they don't
Isolate NAS (through VLAN, probably ?) to limit access to only Alias _Laptops
Anyone wants to journey along ?
Still not ...
I have full access to the NAS (on LAN1) from Laptop1 (on LAN1)
But none of the others have access to it (neitehr ping nor GUI)
I've added a floating rule "Pass" source Alias "_Laptops" (All laptops IPs) to alias "_NAS" any proto any ports, but still can't access
I don't understand ...
Did you recheck all your FW rules, including those automatics ?
Quote from: erica.vh on October 09, 2024, 12:32:05 AM
Did you recheck all your FW rules, including those automatics ?
Not much to be seen there, only the 22 automatic rules which can't be modified nor deleted, and the two clones of the LAN1 autogenerated "Allow all" "IN"
everywhere I search on the internet it says that by default all LAN can communicate between themselves with these standards rules, so there must be something else, but I don't know where to start searching
And when I inspeact the rules by filtering the IP, it only says allow all
I've lost track on this thread. What is the problem and what is the current setup where it manifests itself?
For instance out of the blue comes what seems a VLANs setup. Please describe it, including the setup in OPN and your managed switch for it.
Quote from: cookiemonster on October 10, 2024, 09:58:12 AM
I've lost track on this thread. What is the problem and what is the current setup where it manifests itself?
For instance out of the blue comes what seems a VLANs setup. Please describe it, including the setup in OPN and your managed switch for it.
Good morning,
Replying from the office, I should be ashamed :-p
- LAN1 192.168.101.101/24 => Swicth1
-- Laptop1 192.168.101.102
-- NAS1 192.168.101.111, 19.168.101.112
- LAN2 192.168.102.101/24 => WiFi AP
- LAN3 192.168.103.101/24 => Swicth2
-- Laptop4 192.168.103.102
-- NAS2 192.168.103.111, 192.168.103.112
Alias _Laptop (all laptops IPs) existing but not used/set in any rules (for later)
Alias _NAS (all NAS IPs) existing but not used/set in any rules (for later, as I will restrict WAN access to just 1hr/week for updates)
FW rules all default
I dropped the idea (Although I would have loved it) of VLAN, for I only have unmanaged switches and it seems way too complicated to get my NAS on a VLAN without it.
NAS FW disabled, NAS AV disabled, NAS AMware disabled
Laptop1 can access NAS1 but can't access NAS2
Laptop4 can access NAS2 but can't access NAS1
Need: All LAN to access all LAN, or all laptops to access all NAS
All devices involved have a /24 netmask (255.255.255.0) and the .101 address in the respective network as their default gateway?
Check on the devices themselves, like e.g. `ipconfig /all` on Windows.
Quote from: Patrick M. Hausen on October 10, 2024, 01:02:38 PM
All devices involved have a /24 netmask (255.255.255.0) and the .101 address in the respective network as their default gateway?
Check on the devices themselves, like e.g. `ipconfig /all` on Windows.
Yes sir, thank to this forum (and to you for a great deal) I have a very robust and simple set up
All interfaces are 192.168.10x.101/24, with 192.168.10x.102-112 static, and 192.168.10x.116-122 DHCP
All devices are set to full auto
All devices have access to WAN and outside, IN/OUT
The only blockage I just can't find is why they can't access to each-other
Allias _Laptop is IPs only, and not used in any rules (yet)
Allias _NAS is IPs only, and not used in any rules (yet) (Although I see in some report that these IPs couldn't be resolved, I do have access to the main one's GUI on HTTPS 192.168.101.112
Allias _Printers is IPs only, and not used yet
Aliases are not restricted to same subnet only, right ? (i.e: _Laptops 192.168.101.102, 192.168.102.106)
Then show the DHCP settings of LAN1, LAN2 and LAN3, and the firewall rules, too, please.
On more quick shot: what do you mean by "access"? Ping not working? If ping from a PC to a NAS is working but you say you cannot "access" it - do you mean ... like ... browse in "Network Neighborhood"? That does not work across different routed interfaces :) You need to manually map a drive using the NAS' IP address or DNS name.
HTH,
Patrick
Yes I was going to ask that, what is meant by can not access.
It seems each has two IPs on the same network. The device is probably only listening on one.
Quote from: Patrick M. Hausen on October 10, 2024, 02:49:33 PM
Then show the DHCP settings of LAN1, LAN2 and LAN3, and the firewall rules, too, please.
On more quick shot: what do you mean by "access"? Ping not working? If ping from a PC to a NAS is working but you say you cannot "access" it - do you mean ... like ... browse in "Network Neighborhood"? That does not work across different routed interfaces :) You need to manually map a drive using the NAS' IP address or DNS name.
HTH,
Patrick
Sure thing, I'll get a copy of all these automatic rules when I get back home.
No access as in no ping, and therfore of course no GUI
Laptop4 on LAN3 ping 192.168.101.111 or 112 = 100% loss
Laptop1 on LAN1 ping 192.168.103.111 or 112 = 100% loss
DHCP setting is:
LAN1 192.168.101.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
LAN2 192.168.102.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
LAN3 192.168.103.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
As suggested here, the static address are outside the DHCP range: and the NAS gets the .111 and .112
The NAS themselves have no DHCP (since there is no devices connected to it)
The NAS network setting, since getting a static address from teh router, is set to automatic (auto IP, auto DNS, etc ..) same as all other devices.
NB: At the very begining of this thread, I couldn't even access the GUI within the same subnt,
But thks to QNAP forum, someone suggested to un-trunk the two interace on the NAS side, and it worked right away
Quote from: MarieSophieSG on October 10, 2024, 06:40:48 PM
Sure thing, I'll get a copy of all these automatic rules when I get back home.
Not the automatic ones - the three "allow" rules you placed on these interfaces by copying.
Quote from: MarieSophieSG on October 10, 2024, 06:40:48 PM
No access as in no ping, and therfore of course no GUI
Laptop4 on LAN3 ping 192.168.101.111 or 112 = 100% loss
Laptop1 on LAN1 ping 192.168.103.111 or 112 = 100% loss
I see.
Quote from: MarieSophieSG on October 10, 2024, 06:40:48 PM
DHCP setting is:
LAN1 192.168.101.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
LAN2 192.168.102.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
LAN3 192.168.103.101/24 DHCP 116 - 122 (whiles static addresses are in the .102-.115 range)
A bit unusual but nothing looks broken about it. I mean most people including myself would give the .1 to OPNsense and have dynamic pool of e.g. .100 to .254 or similar. Why so small?
Quote from: MarieSophieSG on October 10, 2024, 06:40:48 PM
As suggested here, the static address are outside the DHCP range: and the NAS gets the .111 and .112
The NAS themselves have no DHCP (since there is no devices connected to it)
The NAS network setting, since getting a static address from teh router, is set to automatic (auto IP, auto DNS, etc ..) same as all other devices.
OK - finally something that does look fishy ;) Why two addresses for the NAS? You cannot connect two interfaces to the same network. Won't work as you now experience.
If it's one port for the NAS and one dedicated IPMI port, fine, of course. But if it's two NAS ports - never connect both to a single network.
Quote from: cookiemonster on October 10, 2024, 04:04:41 PM
Yes I was going to ask that, what is meant by can not access.
It seems each has two IPs on the same network. The device is probably only listening on one.
On my previous router with a very (very) basic FW, the router was connected with both RJ45, but trunked on device side, so ys, only 1 IP to access it,
But since on OPNsense the port-trunking didn't work (I couldn't access the NAS' GUI) they are now separated (not trunked) and I can access the NAS' GUI from either or both at the same time (i.e: in two different browser tab)
But only from within the sub-net
Quote from: Patrick M. Hausen on October 10, 2024, 06:51:38 PM
Quote from: MarieSophieSG on October 10, 2024, 06:40:48 PM
As suggested here, the static address are outside the DHCP range: and the NAS gets the .111 and .112
The NAS themselves have no DHCP (since there is no devices connected to it)
The NAS network setting, since getting a static address from teh router, is set to automatic (auto IP, auto DNS, etc ..) same as all other devices.
OK - finally something that does look fishy ;) Why two addresses for the NAS? You cannot connect two interfaces to the same network. Won't work as you now experience.
If it's one port for the NAS and one dedicated IPMI port, fine, of course. But if it's two NAS ports - never connect both to a single network.
The NASes have two network interfaces,
NAS1 has 2x 2,5 GbE and NAS2 has 2x 1GbE, with a failover (if one is down, or one is overloaded, traffic goes to the other)
Each independant from the other, so I can, if I want, connect 1 laptop to 192.168.101.111 as root, and 1 laptop to 192.168.101.112 as user
Firewall: Rules: IGC0_SWITCH1_ETH1_CAT7green
(No Category)Block WAN at night
Protocol Source Port Destination Port Gateway Schedule Description
Automatically generated rules
IPv4+6 * IGC1_MoDem_ETH2_CAT8black net * _QNAP * * QNAP_Update Allow QNAP To/From WAN
IPv4 * IGC0_SWITCH1_ETH1_CAT7green net * * * * * Default allow LAN to any rule
IPv6 * IGC0_SWITCH1_ETH1_CAT7green net * * * * * Default allow LAN IPv6 to any rule
pass block reject log in first match
pass (disabled) block (disabled) reject (disabled) log (disabled) out last match
Active/Inactive Schedule (click to view/edit)
Enable Enable DHCP server on the IGC0_SWITCH1_ETH1_CAT7green interface
Deny unknown clients
If this is checked, only the clients defined below will get DHCP leases from this server.
Ignore Client UIDs
By default, the same MAC can get multiple leases if the requests are sent using different UIDs. To avoid this behavior, check this box and client UIDs will be ignored.
Subnet 192.168.0.0
Subnet mask 255.255.0.0
Available range 192.168.0.1 - 192.168.255.254
Range
from to
192.168.101.116
192.168.101.122
Additional Pools
Pool Start Pool End Description
If you need additional pools of addresses inside of this subnet outside the above Range, they may be specified here.
That's ugly ! is there a better way to present it ?
There is nt much to be seen here besides what I wrote earlier about static/dhcp,
or is it the lease you wanna see ?
Firewall: Rules: IGC3_SWITCH2_ETH4_CAT7white
(No Category)Block WAN at night
Protocol Source Port Destination Port Gateway Schedule Description
Automatically generated rules
IPv4+6 TCP/UDP * * sshlockout _Anti_Lockout_Ports * * Anti Lockout Rules
IPv6 * IGC3_SWITCH2_ETH4_CAT7white net * * * * * Default allow LAN IPv6 to any rule
IPv4 * IGC3_SWITCH2_ETH4_CAT7white net * * * * * Default allow LAN to any rule
Delete
Quote from: MarieSophieSG on October 10, 2024, 11:43:12 PM
The NASes have two network interfaces,
NAS1 has 2x 2,5 GbE and NAS2 has 2x 1GbE, with a failover (if one is down, or one is overloaded, traffic goes to the other)
Each independant from the other, so I can, if I want, connect 1 laptop to 192.168.101.111 as root, and 1 laptop to 192.168.101.112 as user
This is fundamentally impossible in networking. A system cannot have two interfaces in a single network. Period.
One possible cause of your problems.
Quote from: MarieSophieSG on October 10, 2024, 11:44:45 PM
That's ugly ! is there a better way to present it ?
There is nt much to be seen here besides what I wrote earlier about static/dhcp,
or is it the lease you wanna see ?
I do not get what you mean. What I want from you is something like this:
(https://forum.opnsense.org/index.php?action=dlattach;topic=43205.0;attach=38573;image)
Why is it so difficult for you to show screen shots of your configuration when asked?
Quote from: Patrick M. Hausen on October 11, 2024, 01:00:14 AM
Quote from: MarieSophieSG on October 10, 2024, 11:44:45 PM
That's ugly ! is there a better way to present it ?
There is nt much to be seen here besides what I wrote earlier about static/dhcp,
or is it the lease you wanna see ?
Why is it so difficult for you to show screen shots of your configuration when asked?
Because in a previous post/thread you said: - "no screenshot, I want the code"
https://ibb.co/bdKRXLT (http://lan1)
https://ibb.co/PG3VFPy (http://lan3)
Quote from: MarieSophieSG on October 11, 2024, 01:26:08 AM
Because in a previous post/thread you said: - "no screenshot, I want the code"
For command output, of course. Not for UI things :) Sorry about the confusion.
But please attach them to your post. The links don't open for me.
Quote from: Patrick M. Hausen on October 11, 2024, 07:12:08 AM
Quote from: MarieSophieSG on October 11, 2024, 01:26:08 AM
Because in a previous post/thread you said: - "no screenshot, I want the code"
For command output, of course. Not for UI things :) Sorry about the confusion.
But please attach them to your post. The links don't open for me.
TY, and no, don'T be, I am the one sorry for my autism makes this situation/confusion quite usual to me :/
Attached to the post ? I'm using an external link on purpose because I didn't find an "attach picture" (as in: hosted on this opnsense.org server)
I will look again
Found it !
it's not in the button menu above, it's a link sub-menu below !
OK, the first of the IGC0 rules is useless. On IGC0 there will never be a packet coming IN with a source of IGC1 network. That rule needs to be on the IGC1 interface. Rules always go where the initial packet of the connection first hits the firewall on the way in to some firewall interface. "In" and "out" are viewed from the firewall interface, not a human definition of e.g. "Internet is outside" and "home is inside" or some such.
But the rule does not hurt. It just never matches.
The following two rules allow any system with a source address in the IGC0 network to contact anything else - all systems on all other interfaces, everything on the Internet, etc.
So there is a PC on IGC0 and some other system (NAS?) on a different interface and the PC cannot ping the NAS? Correct?
We need to find out why that is the case because the rules clearly allow that.
The only thing I can think of at the moment: edit the rule on IGC0 for IPv4 - is there an explicit "Gateway" setting? If yes, what is it and why? ;)
Quote from: Patrick M. Hausen on October 11, 2024, 12:33:28 PM
OK, the first of the IGC0 rules is useless. On IGC0 there will never be a packet coming IN with a source of IGC1 network. That rule needs to be on the IGC1 interface. Rules always go where the initial packet of the connection first hits the firewall on the way in to some firewall interface. "In" and "out" are viewed from the firewall interface, not a human definition of e.g. "Internet is outside" and "home is inside" or some such.
But the rule does not hurt. It just never matches.
Good morning,
OK, so as soon as I get my GUI access back, I will move this one to IGC1_WAN (still with "IN", right ?)
Quote from: Patrick M. Hausen on October 11, 2024, 12:33:28 PM
The following two rules allow any system with a source address in the IGC0 network to contact anything else - all systems on all other interfaces, everything on the Internet, etc.
So there is a PC on IGC0 and some other system (NAS?) on a different interface and the PC cannot ping the NAS? Correct?
We need to find out why that is the case because the rules clearly allow that.
Yes, exactly.
Devices on LAN1 can't reach devices on LAN2, LAN3
Device on LAN3 can't reach devices on LAN1
Quote from: Patrick M. Hausen on October 11, 2024, 12:33:28 PM
The only thing I can think of at the moment: edit the rule on IGC0 for IPv4 - is there an explicit "Gateway" setting? If yes, what is it and why? ;)
Nope, all gateway are "default"
Some progress ...
I now can access the NAS GUI (192.168.101.111 & 112) from 192.168.103.102
I've added rules to the FW (screenshot)
It's definitely not ideal, and I still can't map network drives, but at least I can access/download/upload files to NAS1 from LAptop4
Edit: no longer ... the dumb-noob me did some other tweak and I lost access to the GUI of the NAS ... working on reverting one by one ...
Why the same rule (it seems) twice?
Quote from: Patrick M. Hausen on October 12, 2024, 02:29:00 PM
Why the same rule (it seems) twice?
One is on INT IGC0, the other is on INT IGC3, each with an IN/OUT
Device on IGC0 => IN -> FW (INT IGC0) -> OUT => IGC3
Device on IGC3 => IN -> FW (INT IGC0) -> OUT => IGC0
You cannot have a rule in some interface and out some other in a single rule.
You need one rule on interface IGC0 like this (replace my "LAN" with your "IGC0" or whatever you name it):
(https://forum.opnsense.org/index.php?action=dlattach;topic=43205.0;attach=38634;image)
to permit all devices on IGC0 to access the Internet and all other VLANs.
Similar rules on your other two interfaces.
1. You do not need more than one rule per interface.
2. You do not need and should not create "out" rules.
3. You do not need and should not create floating rules.
Place one rule on each interface. That's it. See second screenshot. This is where the single rule for "LAN" goes:
(https://forum.opnsense.org/index.php?action=dlattach;topic=43205.0;attach=38636;image)
.
Quote from: Patrick M. Hausen on October 12, 2024, 03:52:41 PM
You cannot have a rule in some interface and out some other in a single rule.
You need one rule on interface IGC0 like this (replace my "LAN" with your "IGC0" or whatever you name it):
to permit all devices on IGC0 to access the Internet and all other VLANs.
Similar rules on your other two interfaces.
1. You do not need more than one rule per interface.
2. You do not need and should not create "out" rules.
3. You do not need and should not create floating rules.
Place one rule on each interface. That's it. See second screenshot. This is where the single rule for "LAN" goes:
Ok, so ...
Delete all floating rules, check !
Never use floating rules, check ! (but then why is there the option ?)
Never make a rule with OUT, check ! (But then why the option ?)
Never make a rule with IN&OUT, check ! (this option seems to be only available in floating)
Make one rule per interface per direction, check !
-=-=-=--=-=-=-=-=--=-=--=---=-=
The example you are showing are the LAmbda rule from initial set-up, cloned from LAN1, ... all my interfaces have one ! (well, two, actually, one v4 and one v6)
And yet I still can't access other LANs (even after my Fresh re-install this morning)
That's why I'm trying to pocking around and create rules using the available options (rather than just asking step by step, so as to learn a bit on the way)
Laptop1 (LAN1 .101.103) can't access/ping NAS2 (LAN3 .103.112); can't even access LAN3 interface (192.168.103.101)
Laptop4 (LAN3 .103.103) can't access/ping NAS1 (LAN1 .101.112); can't even access LAN1 interface (192.168.101.101)
Idem for LAN2
So I'm lost as it's "supposed" to have access out of the box, and I did a fresh re-install this morning ... (I'm gonna cry)
Here is my beautiful latest diagram ! I have added missing colours and flowers and puppy and trolls and gremlins :-p
Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.
A system cannot have two interfaces in a single network. Networking 101.
Quote from: Patrick M. Hausen on October 12, 2024, 06:33:42 PM
Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.
A system cannot have two interfaces in a single network. Networking 101.
Done, and OPNsense rebooted, still the same,
Laptop1 (LAN1, .101.103) has no access to NAS2 (LAN3, .103.112) not even ping the LAN3, 103.101 interface,
Laptop4 (LAN3, .103.103) has no access to NAS1 (LAN1, .101.112) not even ping the LAN1, 101.101 interface
From the OPNsense box, all LANs are pingable
From the box, using a source address, .101.103 & .103.103 I can't ping neither (see image) it's as if the NAS was offline ... but since I can't even ping (from devices) the interface, it's not surprise
Quote from: Patrick M. Hausen on October 12, 2024, 06:33:42 PM
Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.
A system cannot have two interfaces in a single network. Networking 101.
Bringing it back here as it's not related to the other thread (to tidy it up)
I apologize if I sounded pretentious or know-it all, or such, it's absolutely not the case, it's just that my brain has a hard time processing what doesn't make sense based on what I see in front of me
I thank you for the link and I certainly will read it to be able to go to bed less stupid tonight.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
But *I am* connected to both interface ...
on two different tab of the same browser (same user)
On two different browsers (same or two diff. user)
If I wanted, I could connect with three diff. browsers using each one of the three users
I use .111 for local access, (through browser, for admin work)
and .112 for applications access (i.e: mapping network folder, Streaming, ...)
Before OPNsense, I was using interface binding to get only one IP out with a fail-over, but trunking doesn't work with OPNS
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-
Quote from: MarieSophieSG on Today at 06:56:15 pm
But *I am* connected to both interface ...
-=--=-=-
Then everything works as you intend it to do and we can close all the threads. Right?
The fact that you can plug two interfaces of one box into the same switch and have both get an IP address via DHCP does in no way confirm that this is in any way a supported topology in IP networking. It's not.
Read this article, please:
https://www.truenas.com/community/resources/multiple-network-interfaces-on-a-single-subnet.45/
If you think you know better than me, I am obviously in no position to help you.
Kind regards,
Patrick
-=-=-=-=-=-=-=-=-=-=-=-==--
Quote from: Patrick M. Hausen on Today at 07:30:26 pm
If you think you know better than me, I am obviously in no position to help you.
-=-=-
You know I don't, and I certainly don't even pretend to, I have many flaws (based on what personal I already said , but not this one.
It's just what I have in front of me versus what you say makes me bug for I don't understand how a device meant to be used with two interfaces, each it's own MAC, etc ... and all managed internally by the NAS itself, couldn't just do that.
Anyway, I've unplugged one RJ45 on each, no problem, as I can't do the transfers/streaming I want for now, so just one is plenty enough for the tests (on the other thread)
Quote from: Patrick M. Hausen on October 12, 2024, 06:33:42 PM
Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.
A system cannot have two interfaces in a single network. Networking 101.
https://www.truenas.com/community/resources/multiple-network-interfaces-on-a-single-subnet.45/
Interesting, thank you again for the link.
And as the person concludes, he won't debate the subject, and so won't I :)
Funny enough they mention LACP/LAGG (Aggregating, Binding, Truncking) for xNIX system while unbound interfaces would work on windows, as for me the only reason I'm not using the LACP is because it doesn't work on my OPNsense ... while it works (worked) on windows !
Anyway, now I have only 1 RJ45 connected, that's it.
LACP works perfectly on OPNsense.
All my internal networks are VLANs on top of a LAGG connected to a Mikrotik CRS326-24G-2S+IN via 2x fiber 10G.
All my NAS systems (1 TrueNAS CORE - FreeBSD, 1 TrueNAS SCALE - Linux) are connected via LACP to that same switch.
FreeBSD has supported LACP for decades and it is rock solid. You must use a managed switch that also supports LACP on the other end, of course.
Quote from: Patrick M. Hausen on October 12, 2024, 09:40:09 PM
LACP works perfectly on OPNsense.
All my internal networks are VLANs on top of a LAGG connected to a Mikrotik CRS326-24G-2S+IN via 2x fiber 10G.
All my NAS systems (1 TrueNAS CORE - FreeBSD, 1 TrueNAS SCALE - Linux) are connected via LACP to that same switch.
FreeBSD has supported LACP for decades and it is rock solid. You must use a managed switch that also supports LACP on the other end, of course.
It just didn't work on mine, but as soon as I have this access problem solved, I will sure give it another try, as it worked before
Just not on this setup, not "out of the box" ... with port-truncking, I couldn't even ping the NAS on its own LAN, while now with the two interface separated, at least I can acces it from its own LAN
So, what or where do you think I should be looking for that LAN-LAN not talking ?
It might be the peable in the greabox that leads to all the other problems ?
(well, not the IDS/IPS problem I just had and posted about, but all the others)
Hum ...
I must be doing something wrong with the Ping as I can't even ping my own interface from it ??
Laptop1 ping perfectly LAN1 from the device, but the PING GUI suggest it doesn't go through ?
EDIT: I inverted host/source
Here is a compete list of 15 pings,
From host Laptop, from host LAN, to NAS1, to NAS2
So within OPNsense:
Laptop4 (LAN3) can ping LAN1
Laptop4 (LAN3) can ping NAS1 (LAN1)
But from the device,
Laptop4 can not ping LAN1
Laptop4 can not ping NAS1
I went back to triple check the rules, the only diff. nbetween the lamda Allow-All clones from LAN1 is that on LAN2 and LAN3 the IPv6 was above IPv4 (corrected now) bot other than that, all the same
Patch:
After I did a full re-Install, since "by default" I still don't have communication between LANs, and because computer4 needs access to NAS1 more than computer1, I moved the NAS1 to LAN3
Now computer4 has access to it, but Computer1 doesn't.
Still with the LAN-LAN no-communication problem
BTW' even though ClamAV was disabled and removed, while doing the Full-Reinstall I saw ClamAV messages in the log, meaning disabling ClamAV didn't actually remove it, keeping it "running/ -although not filtering- in the background
For avoidance of doubt, back to square one.
WorkBench, screen+keyboard:
- Full re-install 24.7 (ZFS strip, all default)
On GUI:
- Named all interface
- Set static IPv4 for each
- Enabled DHCP on each
- Set DNS IPs
- cloned lambda rules "allow all" from LAN1 to LAN2 and LAN3
- All devices on DHCP (no static)
- NOT updated to 27.4.6
- NOT installed any plugin
- NOT set IDS/IPS or blocklist and such
- NOT tweaked any other way
=> tests:
All devices access the Internet
Laptop1 accesses LAN1 interface 192.168.101.101
Laptop1 (LAN1) does not access LAN2 .102.101 LAN3 .103.101 interfaces
Tablet (LAN2) does not access LAN2 interface 192.168.102.101
Tablet (LAN2) does not access LAN1 .101.101 LAN3 .103.101 interfaces
Laptop4 accesses LAN3 interface 192.168.103.101
Laptop4 (LAN3) does not access LAN1 .101.101 LAN2 .102.101 interfaces
=> PING (from inside OPNsense) tests
Laptop1 (LAN1) can not ping LAN2 LAN3
Laptop4 (LAN3) can ping LAN1 and LAN2
Let's go one at the time i.e. diagnose between two interfaces only. Then move to the next, OK ?
OPN clean slate, basic setup - check.
What are Lambda rules BTW ?
Now then. Are you able for diagnostics to put a laptop on an interface when diagnosing? I ask because it seems you are doing ping tests from the device itself. It "should" be the same, but when diagnosing is best to not assume. Unless I'm imagining it incorrectly, if you are putting OPN on a workbench, then the tests aren't going to be true reflection because the interfaces being tested will be down. Please inform how are they "UP" in a workbench. If they are, then the request to put a laptop on it would not be a problem, right?
Quote from: cookiemonster on October 14, 2024, 10:47:48 PM
Let's go one at the time i.e. diagnose between two interfaces only. Then move to the next, OK ?
OPN clean slate, basic setup - check.
What are Lambda rules BTW ?
Now then. Are you able for diagnostics to put a laptop on an interface when diagnosing? I ask because it seems you are doing ping tests from the device itself. It "should" be the same, but when diagnosing is best to not assume. Unless I'm imagining it incorrectly, if you are putting OPN on a workbench, then the tests aren't going to be true reflection because the interfaces being tested will be down. Please inform how are they "UP" in a workbench. If they are, then the request to put a laptop on it would not be a problem, right?
The "lambda" rules are the two extras (extras from the automatic ones) generated on LAN1 at initial setup
All laptops are on their own interfaces,
Laptop1 (LAN1) can't ping any LAN2 LAN3 interfaces, yet along any devices on these
Laptop4 (LAN3) can't ping any LAN1 LAN2 interfaces, yet along any devices on these
The workbench is only for local install (screen+keyboard) then back to "normal" plugging
Didn't do any test from the workbench, as you rightfully said it would not representative
Still not clear about lambda but let's see. Can you please post screenshot of your LAN1 firewall rules. No link to external sites please. No need to expand the automatic ones yet.
What we would like to do is (ideally) have a laptop on each interface, through a switch on each if that is the current setup, to then do the pings.
We would also enable OPN additional logging if is on defaults:
Firewall: Settings: Advanced | Logging section. We enable to diagnose and then disable as it eats storage.
Quote from: cookiemonster on October 14, 2024, 11:48:56 PM
Still not clear about lambda but let's see. Can you please post screenshot of your LAN1 firewall rules. No link to external sites please. No need to expand the automatic ones yet.
We would also enable OPN additional logging if is on defaults:
Firewall: Settings: Advanced | Logging section. We enable to diagnose and then disable as it eats storage.
Sure ! You will see nothing but the lambda (as in "default", the two pre-set rules for LAN1, right after the automatic ones)
Quote from: cookiemonster on October 14, 2024, 11:48:56 PM
What we would like to do is (ideally) have a laptop on each interface, through a switch on each if that is the current setup, to then do the pings.
That's exactly what I did and reported in the previous
Quote from: cookiemonster on October 14, 2024, 11:48:56 PM
We would also enable OPN additional logging if is on defaults:
Firewall: Settings: Advanced | Logging section. We enable to diagnose and then disable as it eats storage.
Logging are on by default
And the network setup,
Except NAS1 is out and there is no "static" (for now)
So, focusing on "Laptop1 (LAN1) can't ping any LAN2 LAN3 interfaces, yet along any devices on these".
- Your LAN1 on the above statement, trying to find it on your diagram, is igc0 with ip 192.168.101.101/24, right?
- Your firewall rules on it says all in allowed on IPV4 and IPv6. Good.
1) Now, plug your laptop on the switch and should, according to your diagram, get an ip in range 116-122 so why does it have instead 192.168.101.102 ? Static set by you? if so, have you checked you have also entered the correct gateway and netmask ?
2) When you ping for this laptop, do you see the traffic hitting the OPN firewall ? Blocked or allowed, doesn't matter. You need to be able to see the traffic. To do that, you need to enable logging that is disabled by default, and you've done that (it is not on by default). And on your firewall rule screenshot it appears disabled for the rule in particular, and that would override the general setting. Please double check and report.
Quote from: cookiemonster on October 15, 2024, 11:49:33 AM
So, focusing on "Laptop1 (LAN1) can't ping any LAN2 LAN3 interfaces, yet along any devices on these".
- Your LAN1 on the above statement, trying to find it on your diagram, is igc0 with ip 192.168.101.101/24, right?
- Your firewall rules on it says all in allowed on IPV4 and IPv6. Good.
Yes sir, that's exactly it
Quote from: cookiemonster on October 15, 2024, 11:49:33 AM
1) Now, plug your laptop on the switch and should, according to your diagram, get an ip in range 116-122 so why does it have instead 192.168.101.102 ? Static set by you? if so, have you checked you have also entered the correct gateway and netmask ?
That's why I mentioned "no static" (for now) as this IP was (and will be) its static one, therefore out of the DHCP range
For now, it gets a DHCP address .101.116
Quote from: cookiemonster on October 15, 2024, 11:49:33 AM
2) When you ping for this laptop, do you see the traffic hitting the OPN firewall ? Blocked or allowed, doesn't matter. You need to be able to see the traffic. To do that, you need to enable logging that is disabled by default, and you've done that (it is not on by default). And on your firewall rule screenshot it appears disabled for the rule in particular, and that would override the general setting. Please double check and report.
Logging *is* "on" by default, as I haven't touched anything there since re-install
Or maybe what you mean is I need *the only one* that is not "on" by default, the:
Outbound NAT Log packets matched by automatic outbound NAT rulesOK, it is ON now,
Where do I check its results ? in rules statistics ?
Live traffic logging view:
Firewall: Log Files: Live View
Chose the INCOMING interface from the dropdown for first filter. i.e. Interface IS LAN1 or whatever you have called it.
p.s. Strange you have that on after reinstall. Never seen that happen. You ought to remember to disable it later when all is good. It'll chew your storage if the firewall is a busy one.
Quote from: cookiemonster on October 15, 2024, 12:21:06 PM
Live traffic logging view:
Firewall: Log Files: Live View
Chose the INCOMING interface from the dropdown for first filter. i.e. Interface IS LAN1 or whatever you have called it.
p.s. Strange you have that on after reinstall. Never seen that happen. You ought to remember to disable it later when all is good. It'll chew your storage if the firewall is a busy one.
In the live view with filter on .101.116 and .103.101 with PING running (inside OPNs)
Ping results = 100% loss
And live view on LAN1
I don't see the ping from 192.168.101.116 to 192.168.103.101 indeed.
Just to be sure the logging is correct. What I meant before, here in your screenshot, modified to show:
The greyed out i suggest the logging of those rules is disabled. Can you try to enable it?
EDIT:
Why isn't your interface IGC0 on your picture?
In other words, your interface LAN1 maps to IGC0 according to your diagrams. So why is it showing as "LAN1_SWITCH1_Green_ETH1_IGC1". Just a labelling error?
Quote from: cookiemonster on October 15, 2024, 12:50:35 PM
I don't see the ping from 192.168.101.116 to 192.168.103.101 indeed.
Just to be sure the logging is correct. What I meant before, here in your screenshot, modified to show:
The greyed out i suggest the logging of those rules is disabled. Can you try to enable it?
In FW, Settings, Avenced all logging are enabled
Quote from: cookiemonster on October 15, 2024, 12:50:35 PM
EDIT:
Why isn't your interface IGC0 on your picture?
In other words, your interface LAN1 maps to IGC0 according to your diagrams. So why is it showing as "LAN1_SWITCH1_Green_ETH1_IGC1". Just a labelling error?
You are absolutely right, I meant to correct it yesterday but forgot
LAN1 is IGC0, not IGC1 (WAN) corected now
Thanks for the confirmation of the interface names.
Quote from: MarieSophieSG on October 15, 2024, 01:04:36 PM
Quote from: cookiemonster on October 15, 2024, 12:50:35 PM
I don't see the ping from 192.168.101.116 to 192.168.103.101 indeed.
Just to be sure the logging is correct. What I meant before, here in your screenshot, modified to show:
The greyed out i suggest the logging of those rules is disabled. Can you try to enable it?
In FW, Settings, Avenced all logging are enabled
That is not what I asked though. Have you enabled the logging for these rulese on the screenshot? Ah wait, it might be something new. The i is clickable to toggle enable/disable. It's a shortcut to editing the rule and doing it there.
Chances are if you click on edit instead of the shortcut, you'll find the rule has no logging enabled. Check please.
Quote from: cookiemonster on October 15, 2024, 01:23:14 PM
Thanks for the confirmation of the interface names.
Quote from: MarieSophieSG on October 15, 2024, 01:04:36 PM
Quote from: cookiemonster on October 15, 2024, 12:50:35 PM
I don't see the ping from 192.168.101.116 to 192.168.103.101 indeed.
Just to be sure the logging is correct. What I meant before, here in your screenshot, modified to show:
The greyed out i suggest the logging of those rules is disabled. Can you try to enable it?
In FW, Settings, Avenced all logging are enabled
That is not what I asked though. Have you enabled the logging for these rulese on the screenshot? Ah wait, it might be something new. The i is clickable to toggle enable/disable. It's a shortcut to editing the rule and doing it there.
Chances are if you click on edit instead of the shortcut, you'll find the rule has no logging enabled. Check please.
By "i" you mean the "information" icon at the very right of each lines on "live view" ? right ?
If the line appears on live view, doesn't that means the logging is on ?
Keep in mind I'M using 24.7, I didn't do any update (To leave *everything* stock, until I'm told to do so)
There is no logging per rule, only the general settings (for all rules) or maybe I didn't look in the right place ?
please look at the attached screenshot of yours that I modified to point it out.
Quote from: cookiemonster on October 15, 2024, 01:54:44 PM
please look at the attached screenshot of yours that I modified to point it out.
Oh ! yes, right, thank you, I missed it :(
Once you've changed or at least checked this, you can probably see where this is going.
If there is no record on the firewall of your ping hitting it (hence is so important that the logging is right), then you need to check why is that. So it won't be a setting on OPN prevent it it, it will be something more "basic" and we will need to go to the very basics of the interface setup.
The only other thought and if it goes that way, is that I see signs of IPv6 which I do not use. I am not familiar with it, so if there is an IPv6 diagnostic required I'll need to step back.
Quote from: cookiemonster on October 15, 2024, 02:00:33 PM
Once you've changed or at least checked this, you can probably see where this is going.
If there is no record on the firewall of your ping hitting it (hence is so important that the logging is right), then you need to check why is that. So it won't be a setting on OPN prevent it it, it will be something more "basic" and we will need to go to the very basics of the interface setup.
The only other thought and if it goes that way, is that I see signs of IPv6 which I do not use. I am not familiar with it, so if there is an IPv6 diagnostic required I'll need to step back.
I clicked the I-information for both lines on both LAN1 and LAN3 FW rules (color slightly changed to blue"
And went back to live view to get this:
just filter on LAN1 please, and ping from the laptop on that network.
If it is not appearing on the live log, the packet is not hitting the firewall. That simple.
From the laptop, ping LAN3 interface 192.168.103.101
I don't see any such appearing in the log
But I see 192.168.101.116 (Laptop1) trying to hit 192.168.101.127 which is an IP leading to none ... strange to me, but maybe there is a rational ?
Quote from: cookiemonster on October 15, 2024, 02:18:07 PM
just filter on LAN1 please, and ping from the laptop on that network.
If it is not appearing on the live log, the packet is not hitting the firewall. That simple.
Filter to LAN1 only
Laptop1 to ping LAN3 interface,
No line appearing in the log ... (waited a couple of minutes with ping still on)
Then is not getting to OPN.
Worth remembering what is meant to happen:
Devices on the same netwkork i.e. on LAN1 will NOT go _through_ OPN. So all devices on that switch will only go through the switch, port to port. Unless you are enabling some routing funtions on that switch. If it is unmanaged, then is a moot point.
Devices on different networks i.e. on different interfaces of OPN will go from one to the other _through_ OPN. So a packet from say client A on LAN1 192.168.101.116 to LAN3 on 192.168.103.101 will go _through_ OPN and once all is logged in OPN, will appear there.
If it doesn't, then is not hitting OPN.
At that point you must go back to checking physical connections, interface setup on OPN and your switches.
Start breaking up the problem with this overview and go in a systematic manner.
Another suggestion: skip pings for a moment and try other type of connections. iperf, maybe a webserver, etc. Easy with a laptop on each end.
Quote from: cookiemonster on October 15, 2024, 03:13:38 PM
Then is not getting to OPN.
Worth remembering what is meant to happen:
Devices on the same netwkork i.e. on LAN1 will NOT go _through_ OPN. So all devices on that switch will only go through the switch, port to port. Unless you are enabling some routing funtions on that switch. If it is unmanaged, then is a moot point.
Devices on different networks i.e. on different interfaces of OPN will go from one to the other _through_ OPN. So a packet from say client A on LAN1 192.168.101.116 to LAN3 on 192.168.103.101 will go _through_ OPN and once all is logged in OPN, will appear there.
If it doesn't, then is not hitting OPN.
At that point you must go back to checking physical connections, interface setup on OPN and your switches.
Start breaking up the problem with this overview and go in a systematic manner.
Another suggestion: skip pings for a moment and try other type of connections. iperf, maybe a webserver, etc. Easy with a laptop on each end.
Thank you
All connection are good (RJ45 IN, led ON, etc..)
All devices on each Interfaces are reaching out to the Internet through OPN
All Interface set to the same as LAN1 (naming, rules, ..)
DNS is autom
All switches are unmanaged, and WiFI AP router is set to bridge, unmanaged
Worth noting: I have access to NAS2 (LAN3) GUI from Laptop2 (LAN2) !!
This is my work laptop, and the only one device out of all which I don't need/want to access any NAS (or anything else on my network for that matter) ... aaarrgggg wwhhyyyy ?
iperf:
sophie@T440p:~
$ iperf -d 192.168.103.101
WARNING: option -d is not valid for server mode
iperf: ignoring extra argument -- 192.168.103.101
Usage: iperf [-s|-c host] [options]
Try `iperf --help' for more information.
sophie@T440p:~
$ iperf -c 192.168.103.101 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.103.101, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
ERROR: expected reverse connect did not occur
tcp connect failed: Connection timed out
[ 1] local 0.0.0.0 port 0 connected with 192.168.103.101 port 5001
sophie@T440p:~
$
quick check before reading your last post properly. Is the problem to reach only one device on the other network or any and all (that you can tell of course) ?
Quote from: cookiemonster on October 15, 2024, 05:58:19 PM
quick check before reading your last post properly. Is the problem to reach only one device on the other network or any and all (that you can tell of course) ?
Good question (to rule out the network but rather incriminate one host)
So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI
while Laptop4, even though also win10 (on LAN3) can't ping other LANs, I therefore assume it won't reach any devices on these
Ping 192.168.101.101, 100% loss
Ping 192.68.101.116 (Laptop1) 100% loss
Ping 192.168.102.101 100% loss
Ping 192.168.102.116 loss
While the Linux and android devices can't
Laptop1 (LAN1) can't ping Laptop4; Laptop3 can't ping NAS2; can't ping Interfaces (other than its own)
Laptop3 (LAN3) can't ping Laptop1; Laptop4; can ping NAS2 (but can't reach its GUI); can't ping Interfaces (other than its own)
Android devices can't reach anything internal (on any LAN, including their own) neither can they browse to the WiFI AP GUI they are attached to
Laptop4 when booted to Qubes-os neither, but that OS is so secured, so far I didn't even mentioned it for networking tests
Traceroute from Laptop1 to NAS2:
sophie@T440p:~
$ traceroute 192.168.103.118
traceroute to 192.168.103.118 (192.168.103.118), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
Tracert from Laptop2 to NAS2
C:\Users\xxxx>tracert 192.168.103.118
tracing route to NAS653D [192.168.103.118]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.102.101
2 <1 ms <1 ms <1 ms NAS653D [192.168.103.118]
Trace complete.
Right then. First recap.
Quote
So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI
So that tells us your LAN2 has a firewall rule that allows to reach LAN1. One network good.
So we're back to LAN1, right ? LAN3 will be next.
So we need to view the firewall rules on this one. Compare to the known good. Share it here as before (lost track if is the one you shared already). And share your interface setup. Could it be set as not /24 I wonder.
For the purpose of diagnosing the network, leave IoT, mobiles phones, smart toasters, etc. out of the picture. If you have a device with a console like a laptop that you seem to have, just stick with them. Keep it simple.
Once problem solved, then you can start looking at those in the knowledge your firewall and rules are not in the way.
p.s. And on your iperf test. a) you did set the server in listening mode (-s), right? ;b) the error suggests there was no return so could be the missing rule on that interface to get the packets back to sender.
In fact, time to do it in pairs.
Quote from: MarieSophieSG on October 15, 2024, 09:56:48 PM
Traceroute from Laptop1 to NAS2:
sophie@T440p:~
$ traceroute 192.168.103.118
traceroute to 192.168.103.118 (192.168.103.118), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
Tracert from Laptop2 to NAS2
C:\Users\xxxx>tracert 192.168.103.118
tracing route to NAS653D [192.168.103.118]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.102.101
2 <1 ms <1 ms <1 ms NAS653D [192.168.103.118]
Trace complete.
Second one is nice and healthy. Shows hitting the OPN LAN3 interface before going out to the device on LAN2 interface.
The first one is not healthy obviously but is not reaching the interface. You need to check the interface second. First, the network stack of that laptop. Suggest you disable and re-enable to recreate the routes when it gets a fresh dhcp lease. If set manually, verify those and the gateway you have set. I thought I had mentioned that before.
Quote from: cookiemonster on October 15, 2024, 09:59:20 PM
Right then. First recap.
Quote
So far, apparently, only 1 Windows computer can reach out to other LANs
Laptop2 (LAN2) can ping NAS2 (LAN3) and browse to its GUI, can ping any LANs interface (.101.101; .102.101; .103.101)
Laptop2 can reach LAN2 interface, and it can reach WiFI AP GUI
So that tells us your LAN2 has a firewall rule that allows to reach LAN1. One network good.
So we're back to LAN1, right ? LAN3 will be next.
So we need to view the firewall rules on this one. Compare to the known good. Share it here as before (lost track if is the one you shared already). And share your interface setup. Could it be set as not /24 I wonder.
I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
Based on what ? what is the parameters that leads to such decision ?
AFAIK, all three LANs are all the same, all with the 20-ish automatic rules and the two initial extras (to avoid "lambda :-p ) ones
I will try a side by side comparison then ...
All the exact same 23 rules, except LAN1 has a 24th self anti-lockout
Quote from: cookiemonster on October 15, 2024, 09:59:20 PM
For the purpose of diagnosing the network, leave IoT, mobiles phones, smart toasters, etc. out of the picture. If you have a device with a console like a laptop that you seem to have, just stick with them. Keep it simple.
Once problem solved, then you can start looking at those in the knowledge your firewall and rules are not in the way.
Sure thing !
Quote from: cookiemonster on October 15, 2024, 09:59:20 PM
p.s. And on your iperf test. a) you did set the server in listening mode (-s), right? ;b) the error suggests there was no return so could be the missing rule on that interface to get the packets back to sender.
In fact, time to do it in pairs.
iperf was set as shown in the code: -c (client) then IP, then -d (double / return)
The -s option is for serveur, taht's why I didn't use it, will try that
sophie@T440p:
$ iperf -s 192.168.103.101 -d
WARNING: option -d is not valid for server mode
iperf: ignoring extra argument -- 192.168.103.101
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
Been in that state for about 5min now ...
Quote from: cookiemonster on October 15, 2024, 10:07:47 PM
Second one is nice and healthy. Shows hitting the OPN LAN3 interface before going out to the device on LAN2 interface.
The first one is not healthy obviously but is not reaching the interface. You need to check the interface second. First, the network stack of that laptop. Suggest you disable and re-enable to recreate the routes when it gets a fresh dhcp lease. If set manually, verify those and the gateway you have set. I thought I had mentioned that before.
All clients are DHCP (for now) since the fresh re-install.
Network boot is always disabled on my devices (never use it, so I took a habit to remove it from the boot options)
> I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
They are logically identical but you might need to go and change the interface.
So say you have a rule on LAN1 interface. Then you clone it to use it on LAN2. You should then verify that the new rule has LAN2 as its interface that it applies to. The cloning won't know what you intended to use it for.
Please post them here instead of assuming.
> iperf was set as shown in the code: -c (client) then IP, then -d (double / return)
The -s option is for serveur, taht's why I didn't use it, will try that
This is just a clarification. When you do an iperf test you need two ends, a client -c and a server -s.
Quote from: cookiemonster on October 16, 2024, 10:10:23 AM
> I'm not sure I understand correctly ? Are you suggesting that automatic rules could be different from one LAN to the other ?
They are logically identical but you might need to go and change the interface.
So say you have a rule on LAN1 interface. Then you clone it to use it on LAN2. You should then verify that the new rule has LAN2 as its interface that it applies to. The cloning won't know what you intended to use it for.
You can't modify the automatic ones, so you can't change the interface nor anything
Quote from: cookiemonster on October 16, 2024, 10:10:23 AM
Please post them here instead of assuming.
Not assuming, I did a side-by-side comparison, and did find that they are all 3 the same 23 rules, *except* for LAN1' 24th rule which is the anti-lock-out
All three attached
Thanks they look correct.
>You can't modify the automatic ones, so you can't change the interface nor anything
I was referring to the cloned ones.
So now time to resume diagnosing LAN1.
Test from a device on LAN1 to a device on LAN3 and watch the live logs. Filter only on interface LAN1. Rules enabled logging. What do you see?
Quote from: cookiemonster on October 16, 2024, 12:10:02 PM
So now time to resume diagnosing LAN1.
Test from a device on LAN1 to a device on LAN3 and watch the live logs. Filter only on interface LAN1. Rules enabled logging. What do you see?
Logging are "on" (the i-information is blue-ish)
Live log of LAN1 (attached) during all these pings
Laptop1 ping 192.168.103.101 (LAN3 interface) 100% loss
Laptop1 ping 192.68.103.118 (NAS2 on LAN3) 100% Loss
Laptop1 ping 8.8.8.8 => 0% Loss (all pass)
Laptop1 ping 178.162.131.118 (this forum) 0% loss
Same result, .. log is empty, nothing through the FW
Sophie@T440p:~
$ ping 192.168.103.101
PING 192.168.103.101 (192.168.103.101) 56(84) bytes of data.
^C
--- 192.168.103.101 ping statistics ---
11 packets transmitted, 0 received, 100% packet loss, time 10231ms
sophie@T440p:~
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=115 time=289 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=115 time=287 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=115 time=291 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=115 time=287 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 286.517/288.294/291.042/1.821 ms
sophie@T440p:~
$ ping 178.162.131.118
PING 178.162.131.118 (178.162.131.118) 56(84) bytes of data.
64 bytes from 178.162.131.118: icmp_seq=1 ttl=49 time=427 ms
64 bytes from 178.162.131.118: icmp_seq=2 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=3 ttl=49 time=440 ms
64 bytes from 178.162.131.118: icmp_seq=4 ttl=49 time=425 ms
64 bytes from 178.162.131.118: icmp_seq=5 ttl=49 time=429 ms
^C
--- 178.162.131.118 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 425.294/429.298/440.282/5.641 ms
sophie@T440p:~
$
EDIT: As I changed yesterday the name of the interface (correction from IGC1 to IGC0) the filter was wrong, here is the correct one
On WAN I changed IPv6 from DHCP to None
Then same pings
No change, I still have IPv6 lines, but overall I have way less lines
Edit:
I went back to WAN to set IPv6 to DHCP, then went to each LAN to set IPv6 to None, and then back to WAN to set IPv6 to None
No change, I still have IPv6 lines in the log ?
If I simplify, is this correct?
Problem statement: devices on LAN1 can not get to devices on LAN3. Maybe to more but one problem at the time.
Devices on LAN1 get an ip in the range 192.168.101.0/24.
The allow all rule seems to be allowing to reach the GUI of OPN but I see a lot of IPv6.
Please show the LAN1 interface setup. A screenshot please.
And a reminder. I don't know IPv6. Are you comfortable disabling it, or do you prefer someone else familiar with it to try and help?
Quote from: cookiemonster on October 16, 2024, 10:53:02 PM
If I simplify, is this correct?
Problem statement: devices on LAN1 can not get to devices on LAN3. Maybe to more but one problem at the time.
Devices on LAN1 get an ip in the range 192.168.101.0/24.
Yes sir, this is correct
Quote from: cookiemonster on October 16, 2024, 10:53:02 PM
The allow all rule seems to be allowing to reach the GUI of OPN but I see a lot of IPv6.
Please show the LAN1 interface setup. A screenshot please.
And a reminder. I don't know IPv6. Are you comfortable disabling it, or do you prefer someone else familiar with it to try and help?
Two posts above, I already disabled it :)
But I still see IPv6 lines in the log ?
keep going, want to see the rest of the config on that page please.
Quote from: cookiemonster on October 17, 2024, 10:09:37 AM
keep going, want to see the rest of the config on that page please.
Here you go !
Is it normal that Laptop1 192.168.101.116 is trying to hit 192.168.101.127 and 192.168.101.255 ?
There is nothing there, no device, etc..
Settings look fine.
Interfaces: Settings . What do you have there? We're looking for anything looking abnormal.
Frankly I do not know what might be the problem. It should just work by the way you have done it so it's a wild goose chase right now for what is happening and where. It might not even be in OPN.
Sure your devices' traffic should be appearing on the firewall live view and appears it doesn't.
p.s. the .255 is the broadcast address. 127 no idea. Unless the ARP table on your device is wrong for whatever reason.
Quote from: cookiemonster on October 17, 2024, 11:45:30 AM
Settings look fine.
Interfaces: Settings . What do you have there? We're looking for anything looking abnormal.
Frankly I do not know what might be the problem. It should just work by the way you have done it so it's a wild goose chase right now for what is happening and where. It might not even be in OPN.
Sure your devices' traffic should be appearing on the firewall live view and appears it doesn't.
p.s. the .255 is the broadcast address. 127 not idea. Unless the ARP table on your device is wrong for whatever reason.
Thank you for the information ! (learning every step of the way ...)
Interfaces, settings =>
(time to go to work now)
Settings there seem OK. Only doubt for me is the IPv6 if disabling would help to diagnose but that's only my ignorance of IPv6.
Quote from: cookiemonster on October 17, 2024, 12:47:05 PM
Settings there seem OK. Only doubt for me is the IPv6 if disabling would help to diagnose but that's only my ignorance of IPv6.
More tests ...
The IPv6 is still OFF, and the IPv6 lines I see in the log are actually the MAC of the interfaces in an IPv6 kinda code, so it's all good, IPv6 is indeed disabled, and rules on all interfaces are now down to 18 (was 22) bevause all the IPv6 rules are out (automatic)
Laptop2 (win10), on RJ45 LAN2 .102.116, can access about everything, LAN1+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10), on WiFI LAN2, all the same
Laptop2 (win10) on RJ45 LAN1 .101.118, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop2 (win10) on RJ45 LAN3 .103.119, can access about everything, LAN2+LAN3 interfaces, the WiFI AP (.102.102) the NAS2 (.103.118)
Laptop1 (Linux), on RJ45 LAN1 .101.116, can only access LAN1 interface, no WiFi AP, no NAS2
Laptop1 (Linux), on RJ45 LAN2 .102.118, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2
Laptop1 (Linux), on WiFi LAN2, all the same
Laptop4 (win10) on RJ45 LAN3 .103.116, can access LAN3 interface and NAS2 (103.118), no other LAN1+LAN2 interface, no WiFi AP (.102.102)
Laptop4 (win10) on WiFi LAN2 .102.120, can access LAN2 interface and WiFI AP (.102.102) no other LAN1+LAN3 interface, no NAS2
On the Log for each Interfaces, the only hit I see are .103.118 (NAS2) reaching out, and Laptop2 (with respectives IP depending on where it's connected) but no hit for the other two Laptops when I ping or when I access a website (i.e: this forum)
So it's probably not an OPNsense problem, and not about the OS, but it might be about the devices themselves ? (based on just one -Laptop2- being able to reach all)
Damned ... Where do I go from there ?
What setup these two may have that prevent OPNsense to see them and route them ?
Knowing that they all have access outside ...
The main and obvious difference I see is that all my Laptops are ThinkPads T. series , W700DS, T440p, P15-Gen2, T61p, T22, (Those not being "seen" behind OPN) while Laptop2 (The one having no problem on OPNsense) is a HP
Internet results about (ThinkPad problem OPNsense) only reports problem *installing* OPNsense ON ThinkCenters, not plugging ThinkPads behind OPN
While I understand the installation trouble (BSD, ZFS, ...) the FW itself should only be managing MAC and IPs, right ? it should not be device specific ?
Should I try a different FW (Endian, DynFI, ZeroShell) ?
I'm glad to see you have made this progress. I'm unclear if the disablement of ipv6 made the difference.
As for what is still not working, I'm not sure. Hardware and OS should not make a difference depending on where they're connected. They'll be using the same networking stack internally i.e. request dhcp lease, send and receive packets, etc.
That said, I am unsure what is the nature of the problems you are still seeing. When you say "can't access", is a bit broad. Also from a glance at the stated last situation, you probably want to check the laptop networking setup, especially if you made changes to them earlier when testing i.e. setting it manually and overriding dhcp.
Respectfully I suggest to continue methodically. My offer to help directly still stands.