[NOOB] Connecting NAS dble ETH to LAN1 not accessible from LAN3

[Patrick M. Hausen]

You cannot have a rule in some interface and out some other in a single rule.

You need one rule on interface IGC0 like this (replace my "LAN" with your "IGC0" or whatever you name it):



to permit all devices on IGC0 to access the Internet and all other VLANs.

Similar rules on your other two interfaces.

1. You do not need more than one rule per interface.
2. You do not need and should not create "out" rules.
3. You do not need and should not create floating rules.

Place one rule on each interface. That's it. See second screenshot. This is where the single rule for "LAN" goes:

[erica.vh]

.

[MarieSophieSG]

You cannot have a rule in some interface and out some other in a single rule.

You need one rule on interface IGC0 like this (replace my "LAN" with your "IGC0" or whatever you name it):
to permit all devices on IGC0 to access the Internet and all other VLANs.

Similar rules on your other two interfaces.

1. You do not need more than one rule per interface.
2. You do not need and should not create "out" rules.
3. You do not need and should not create floating rules.

Place one rule on each interface. That's it. See second screenshot. This is where the single rule for "LAN" goes:

Ok, so ...
Delete all floating rules, check !
Never use floating rules, check ! (but then why is there the option ?)
Never make a rule with OUT, check ! (But then why the option ?)
Never make a rule with IN&OUT, check ! (this option seems to be only available in floating)
Make one rule per interface per direction, check !
-=-=-=--=-=-=-=-=--=-=--=---=-=
The example you are showing are the LAmbda rule from initial set-up, cloned from LAN1, ... all my interfaces have one ! (well, two, actually, one v4 and one v6)
And yet I still can't access other LANs (even after my Fresh re-install this morning)

That's why I'm trying to pocking around and create rules using the available options (rather than just asking step by step, so as to learn a bit on the way)

Laptop1 (LAN1 .101.103) can't access/ping NAS2 (LAN3 .103.112); can't even access LAN3 interface (192.168.103.101)
Laptop4 (LAN3 .103.103) can't access/ping NAS1 (LAN1 .101.112); can't even access LAN1 interface (192.168.101.101)
Idem for LAN2

So I'm lost as it's "supposed" to have access out of the box, and I did a fresh re-install this morning ... (I'm gonna cry)

Here is my beautiful latest diagram ! I have added missing colours and flowers and puppy and trolls and gremlins :-p
 

[Patrick M. Hausen]

Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.

A system cannot have two interfaces in a single network. Networking 101.

[MarieSophieSG]

Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.

A system cannot have two interfaces in a single network. Networking 101.

Done, and OPNsense rebooted, still the same,
Laptop1 (LAN1, .101.103)  has no access to NAS2 (LAN3, .103.112) not even ping the LAN3, 103.101 interface,
Laptop4 (LAN3, .103.103)  has no access to NAS1 (LAN1, .101.112) not even ping the LAN1, 101.101 interface
From the OPNsense box, all LANs are pingable
From the box, using a source address, .101.103 & .103.103 I can't ping neither (see image) it's as if the NAS was offline ... but since I can't even ping (from devices) the interface, it's not surprise

[MarieSophieSG]

Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.

A system cannot have two interfaces in a single network. Networking 101.

Bringing it back here as it's not related to the other thread (to tidy it up)

I apologize if I sounded pretentious or know-it all, or such, it's absolutely not the case, it's just that my brain has a hard time processing what doesn't make sense based on what I see in front of me
I thank you for the link and I certainly will read it to be able to go to bed less stupid tonight.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
But *I am* connected to both interface ...
on two different tab of the same browser (same user)
On two different browsers (same or two diff. user)
If I wanted, I could connect with three diff. browsers using each one of the three users
I use .111 for local access, (through browser, for admin work)
and .112 for applications access (i.e: mapping network folder, Streaming, ...)
Before OPNsense, I was using interface binding to get only one IP out with a fail-over, but trunking doesn't work with OPNS

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-
Quote from: MarieSophieSG on Today at 06:56:15 pm
But *I am* connected to both interface ...
-=--=-=-
Then everything works as you intend it to do and we can close all the threads. Right?

The fact that you can plug two interfaces of one box into the same switch and have both get an IP address via DHCP does in no way confirm that this is in any way a supported topology in IP networking. It's not.

Read this article, please:
https://www.truenas.com/community/resources/multiple-network-interfaces-on-a-single-subnet.45/

If you think you know better than me, I am obviously in no position to help you.

Kind regards,
Patrick
-=-=-=-=-=-=-=-=-=-=-=-==--
Quote from: Patrick M. Hausen on Today at 07:30:26 pm
If you think you know better than me, I am obviously in no position to help you.
-=-=-
You know I don't, and I certainly don't even pretend to, I have many flaws (based on what personal I already said , but not this one.
It's just what I have in front of me versus what you say makes me bug for I don't understand how a device meant to be used with two interfaces, each it's own MAC, etc ... and all managed internally by the NAS itself, couldn't just do that.
Anyway, I've unplugged one RJ45 on each, no problem, as I can't do the transfers/streaming I want for now, so just one is plenty enough for the tests (on the other thread)

[MarieSophieSG]

Disconnect the second network interface of all of your NAS systems. Set the first one to "automatic" for network configuration, reboot.

A system cannot have two interfaces in a single network. Networking 101.
https://www.truenas.com/community/resources/multiple-network-interfaces-on-a-single-subnet.45/

Interesting, thank you again for the link.
And as the person concludes, he won't debate the subject, and so won't I
Funny enough they mention LACP/LAGG (Aggregating, Binding, Truncking) for xNIX system while unbound interfaces would work on windows, as for me the only reason I'm not using the LACP is because it doesn't work on my OPNsense ... while it works (worked) on windows !

Anyway, now I have only 1 RJ45 connected, that's it.

[Patrick M. Hausen]

LACP works perfectly on OPNsense.

All my internal networks are VLANs on top of a LAGG connected to a Mikrotik CRS326-24G-2S+IN via 2x fiber 10G.
All my NAS systems (1 TrueNAS CORE - FreeBSD, 1 TrueNAS SCALE - Linux) are connected via LACP to that same switch.

FreeBSD has supported LACP for decades and it is rock solid. You must use a managed switch that also supports LACP on the other end, of course.

[MarieSophieSG]

LACP works perfectly on OPNsense.

All my internal networks are VLANs on top of a LAGG connected to a Mikrotik CRS326-24G-2S+IN via 2x fiber 10G.
All my NAS systems (1 TrueNAS CORE - FreeBSD, 1 TrueNAS SCALE - Linux) are connected via LACP to that same switch.

FreeBSD has supported LACP for decades and it is rock solid. You must use a managed switch that also supports LACP on the other end, of course.

It just didn't work on mine, but as soon as I have this access problem solved, I will sure give it another try, as it worked before
Just not on this setup, not "out of the box" ... with port-truncking, I couldn't even ping the NAS on its own LAN, while now with the two interface separated, at least I can acces it from its own LAN

So, what or where do you think I should be looking for that LAN-LAN not talking ?
It might be the peable in the greabox that leads to all the other problems ?
(well, not the IDS/IPS problem I just had and posted about, but all the others)

[MarieSophieSG]

Hum ...
I must be doing something wrong with the Ping as I can't even ping my own interface from it ??
Laptop1 ping perfectly LAN1 from the device, but the PING GUI suggest it doesn't go through ?



EDIT: I inverted host/source
Here is a compete list of 15 pings,
From host Laptop, from host LAN, to NAS1, to NAS2

So within OPNsense:
Laptop4 (LAN3) can ping LAN1
Laptop4 (LAN3) can ping NAS1 (LAN1)

But from the device,
Laptop4 can not ping LAN1
Laptop4 can not ping NAS1

I went back to triple check the rules, the only diff. nbetween the lamda Allow-All clones from LAN1 is that on LAN2 and LAN3 the IPv6 was above IPv4 (corrected now) bot other than that, all the same

[MarieSophieSG]

Patch:
After I did a full re-Install, since "by default" I still don't have communication between LANs, and because computer4 needs access to NAS1 more than computer1, I moved the NAS1 to LAN3
Now computer4 has access to it, but Computer1 doesn't.
Still with the LAN-LAN no-communication problem

BTW' even though ClamAV was disabled and removed, while doing the Full-Reinstall I saw ClamAV messages in the log, meaning disabling ClamAV didn't actually remove it, keeping it "running/ -although not filtering- in the background

[MarieSophieSG]

For avoidance of doubt, back to square one.
WorkBench, screen+keyboard:
- Full re-install 24.7 (ZFS strip, all default)
On GUI:
- Named all interface
- Set static IPv4 for each
- Enabled DHCP on each
- Set DNS IPs
- cloned lambda rules "allow all" from LAN1 to LAN2 and LAN3
- All devices on DHCP (no static)
- NOT updated to 27.4.6
- NOT installed any plugin
- NOT set IDS/IPS or blocklist and such
- NOT tweaked any other way
=> tests:
All devices access the Internet
Laptop1 accesses LAN1 interface 192.168.101.101
Laptop1 (LAN1)  does not access LAN2 .102.101 LAN3 .103.101 interfaces
Tablet (LAN2) does not access LAN2 interface 192.168.102.101
Tablet (LAN2)  does not access LAN1 .101.101 LAN3 .103.101 interfaces
Laptop4 accesses LAN3 interface 192.168.103.101
Laptop4 (LAN3) does not access LAN1 .101.101 LAN2 .102.101 interfaces

=> PING (from inside OPNsense) tests
Laptop1 (LAN1) can not ping LAN2 LAN3
Laptop4 (LAN3)  can ping LAN1 and LAN2

[cookiemonster]

Let's go one at the time i.e. diagnose between two interfaces only. Then move to the next, OK ?
OPN clean slate, basic setup - check.
     What are Lambda rules BTW ?

Now then. Are you able for diagnostics to put a laptop on an interface when diagnosing? I ask because it seems you are doing ping tests from the device itself. It "should" be the same, but when diagnosing is best to not assume. Unless I'm imagining it incorrectly, if you are putting OPN on a workbench, then the tests aren't going to be true reflection because the interfaces being tested will be down. Please inform how are they "UP" in a workbench. If they are, then the request to put a laptop on it would not be a problem, right?

[MarieSophieSG]

Let's go one at the time i.e. diagnose between two interfaces only. Then move to the next, OK ?
OPN clean slate, basic setup - check.
     What are Lambda rules BTW ?

Now then. Are you able for diagnostics to put a laptop on an interface when diagnosing? I ask because it seems you are doing ping tests from the device itself. It "should" be the same, but when diagnosing is best to not assume. Unless I'm imagining it incorrectly, if you are putting OPN on a workbench, then the tests aren't going to be true reflection because the interfaces being tested will be down. Please inform how are they "UP" in a workbench. If they are, then the request to put a laptop on it would not be a problem, right?

The "lambda" rules are the two extras (extras from the automatic ones) generated on LAN1 at initial setup

All laptops are on their own interfaces,
Laptop1 (LAN1) can't ping any LAN2 LAN3 interfaces, yet along any devices on these
Laptop4 (LAN3) can't ping any LAN1 LAN2 interfaces, yet along any devices on these

The workbench is only for local install (screen+keyboard) then back to "normal" plugging
Didn't do any test from the workbench, as you rightfully said it would not representative

[cookiemonster]

Still not clear about lambda but let's see. Can you please post screenshot of your LAN1 firewall rules. No link to external sites please. No need to expand the automatic ones yet.
What we would like to do is (ideally) have a laptop on each interface, through a switch on each if that is the current setup, to then do the pings.
We would also enable OPN additional logging if is on defaults:
Firewall: Settings: Advanced | Logging section. We enable to diagnose and then disable as it eats storage.