OPNsense Forum

English Forums => General Discussion => Topic started by: stanthewizzard on May 27, 2021, 10:11:47 pm

Title: [SOLVED] 2 LAN …. No communication
Post by: stanthewizzard on May 27, 2021, 10:11:47 pm
Hello

I’m totally lost in my own network.

I have network 1 with switch 1
192.168.0.0/24
WAN X.X.X.X
Fine

I have network 2 with switch 2
192.168.1.0/24
WAN Y.Y.Y.Y
Fine

I have switch 0 that link 1 tot 2

What do I need to do to make 192.168.0.0/24 see 192.168.1.0/24 ?
They don’t talk to each other. A route ? A special gateway ?

Many thanks for you help
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 28, 2021, 09:23:31 am
Your networks 1 and 2 have their own /24 subnet each. Given that your switch 0 is the only connection between the 2 networks, and this is just a switch (not a router), that is the reason why there isn't any communication between network 1 and 2.

If you don't want to change the subnets of the networks, then you need a device that has an IP address in each of the 2 networks and forwards packets between the networks. That device would also have to do some NAT. Then you can create routes for each member of the 2 networks, using that device as a gateway to the other network.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 28, 2021, 10:31:05 am
Thanks for your answer

What I did
on FW1 I added a RJ45 to switch 1 with IP 192.168.1.111

On FW2 I added a RJ45 to switch 2 with IP 192.168.0.111

I need a gateway ? A route ?

Thanks again
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 28, 2021, 10:59:13 am
What are FW1 and FW2? This looks like FireWall, but you didn't mention a firewall in your first post. I am not sure if understood what your network looks like.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 28, 2021, 11:33:33 am
OPNSENSE1 (Firewall/Route/gateway)                                 OPNSENSE1 (Firewall/Route/gateway)               
               Switch1                                                                         Switch2
LAN1 192.168.0.0/24                                                              LAN2 192.168.1.0/24


There is an RJ45 between switch 1 and 2

Do you find it clearer ? (again thank you for your help)
:)
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 28, 2021, 03:13:48 pm
You have 2 x OPNSENSE1 in your post, is that intended or just a copy/paste error mistake? Since you mentioned FW1 und FW2, I just assume this is copy/paste. This is the picture i currently have:

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
                            |
                            |
                        [Switch 0]
                            |
                            |
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

OPNsense 1 Port 1 is 192.168.1.111/24
OPNsense 2 Port 1 is 192.168.0.111/24



Now you want your LAN 1 clients to be able to communicate with the LAN 2 clients, right?

I think this won't work just by configuring a route or gateway in OPNsense.

Even if you log into one of the OPNsenses and try to ping the other one, I assume this won't work, because a simple switch wouldn't forward a packet from one subnet into a different subnet. This is at least my understanding of subnets and switches.

Is switch 0 just a simple switch?
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 28, 2021, 03:40:49 pm
The first schema is the correct one
This are 2 geosites linked by the switch 0

Lan 1 client with lan 2 client and vice versa

Switch can't do the job but router from opnsense 1 and 2 should ?

Thanjs

Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 28, 2021, 06:26:47 pm
Quote
Switch can't do the job but router from opnsense 1 and 2 should ?

Your OPNsenses could do the job, yes, but the switch between the OPNsenses prevents that.

Alternative solutions could be:

1) Replace the switch 0 with another router. That can be another OPNsense, but any other Linux box could do the job. The only requirement is to have at least two network interfaces, one for each subnet. That would look like this:

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
                            |
                            |
                         [router]
                            |
                            |
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients


2) Just connect the 2 OPNsenses directly. That would look like this:

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
                 |
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

3) Add one of the OPNsenses to the other network. That would look like this

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
                 ------------
                            |
                            |
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

or this

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
                            |
                            |
                 ------------
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients


Note: At this point i should mention that i am primarily a software developer, not a network admin. Before you take a bigger step (eg. buying new hardware), i recommend to get a confirmation of your plan from another person. I think the above is correct, this is what i would do myself, but i can't guarantee this is correct.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 28, 2021, 06:41:01 pm
Schema 2 is the one I tried

Opnsense 1 is connect to opnsense 2 through switch 0 (dumb switch)

It’s not working… what Am I missing in this situation ?

Thank you again and again
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 29, 2021, 10:28:37 am
Quote
Schema 2 is the one I tried

Opnsense 1 is connect to opnsense 2 through switch 0 (dumb switch)

That would be this:

Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
             [Switch 0]
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

OPNsense 1 Port 1 = 192.168.1.111/24
OPNsense 2 Port 1 = 192.168.0.111/24

OPNsense 1 Port 2 = ?
OPNsense 2 Port 2 = ?

Please confirm this is what you tried.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 29, 2021, 10:35:05 am
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
             [Switch 0]
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

[OPNsense 1 Port 1] - 192.168.0.201
[OPNsense 1 Port 2] - 192.168.1.111

[OPNsense 2 Port 2] -192.168.0.111
[OPNsense 2 Port 1] - 192.168.1.201

Confirmed like this

You are again very kind thanks
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 29, 2021, 11:51:06 am
The issue with the connection between the OPNsenses is, their IP adresses belong to different subnets. You have a physical connection between [OPNsense 1 Port 2] and [OPNsense 2 Port 2], but the switch won't be enough to establish a communication between them.

So instead of those incompatible IP addresses

[OPNsense 1 Port 2] - 192.168.1.111
[OPNsense 2 Port 2] - 192.168.0.111

i suggest to create a dedicated subnet for this connection. If you (for example) change the IP addresses to

[OPNsense 1 Port 2] - 192.168.2.111
[OPNsense 2 Port 2] - 192.168.2.112

then your OPNsenses should be able to communicate with each other. That doesn't connect the clients of LAN 1 and LAN 2, but it's a first step.

The next step would be a simple test. Log into one of the OPNsenses and ping the other one:

192.168.2.111 --ICMP--> 192.168.2.112
or
192.168.2.111 <--ICMP-- 192.168.2.112

You will probably need to add some firewall rules, so the ICMP packets are allowed to leave the sending device and are accepted on the receiving device.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 29, 2021, 02:10:39 pm
I’ll try your solution asap
Keep posted here
 ;D
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 31, 2021, 11:45:06 am
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
             [Switch 0]
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

[OPNsense 1 Port 1] - 192.168.0.201
[OPNsense 1 Port 2] - 192.168.2.111

[OPNsense 2 Port 2] -192.168.2.112
[OPNsense 2 Port 1] - 192.168.1.201

No ping
:(((
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 31, 2021, 03:25:32 pm
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 31, 2021, 05:27:00 pm
Ok
The lan 2 lan was not correctly build by our hoster.

I can ping now 2.111 frol 2.112 and vice versa

I tried to build a gateway and route but no success can’t ping 1.0 from 0.0 and vice versa

Thank you very much again
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 31, 2021, 07:50:47 pm
Code: [Select]
[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
             [Switch 0]
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

[OPNsense 1 Port 1] - 192.168.0.201
[OPNsense 1 Port 2] - 192.168.2.111

[OPNsense 2 Port 2] - 192.168.2.112
[OPNsense 2 Port 1] - 192.168.1.201



Quote
I can ping now 2.111 frol 2.112 and vice versa

Good. Now let's try to access a LAN 1 service from the LAN 2 OPNsense.

Step 1: Provide such a service in LAN 1, that is available on a specific TCP port.

HTTP, FTP, XMPP, doesn't matter, as long as it's TCP. Technically it could be UDP, but i don't want that for testing. Can you do that, running a TCP-based service on a LAN 1 client?

Step 2: Create a firewall rule on OPNsense 1.

interface: [OPNsense 1 Port 1]
action: accept (or pass, i currently don't know that it's called)
logging: yes
protocol: TCP
direction: outgoing
destination address: your LAN 1 client
destination port: your LAN 1 client's service port
source address: any

Side note: At this point i don't care about security, i just want this to work. If security matters for you, you will have to find your own way.

Step 3: Ensure that the service is available to the OPNsense 1.

I assume that you have telnet or netcat available on your OPNsenses, or anything similar that can be used for testing simple TCP connections. Otherwise i can't help you.

Open a shell on OPNsense 1, use telnet/netcat/whatever, connect to the LAN 1 service (the one from step 1). If that doesn't work, i would guess the firewall rule from step 2 is not correct. In that case, take a look into the firewall log of OPNsense 1 and adjust the rule accordingly.

Step 4: Allow to connect to the LAN 1 client's service from the OPNsense 2.

I assume you need one additional fireall rule for [OPNsense 2 Port 2] (outgoing) and one additional firewall rule for [OPNsense 1 Port 2] (incoming). Please enable logging for both rules.

Step 5: Tell OPNsense 2 where to find LAN 1.

I haven't done this on OPNsense, so i can only guess how to do that. Based on https://docs.opnsense.org/manual/gateways.html i think that a static route is what you need.

I feel this is enough for one posting.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on May 31, 2021, 08:22:42 pm
Not sure that I can use firewall rule or nat. Between the 2 lan
I need to have all the traffic to go through lan 2 lan
Title: Re: 2 LAN …. No communication
Post by: Sheldon on May 31, 2021, 08:56:13 pm
In one of my earlier posts i asked
Quote
Now you want your LAN 1 clients to be able to communicate with the LAN 2 clients, right?

and your response was
Quote
Lan 1 client with lan 2 client and vice versa

At that point i thought that i understand what your goal is. But this
Quote
Between the 2 lan I need to have all the traffic to go through lan 2 lan
made me shaking my head. I need more input on this, otherwise i don't understand what your goal is.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on June 01, 2021, 08:26:41 am
The comm between the 2 network need to be transparent (has with a site 2 site vpn … the one I want to discard).
At my knowledge level there is no NAT in private network

I this clearer ?  ;D
Title: Re: 2 LAN …. No communication
Post by: Sheldon on June 01, 2021, 11:22:53 am
Hmm, you didn't mention before that there is a VPN involved? That's a factor, that would have been good to know from the beginning (not on page 2).

I have a feeling that i can't help you any further. I know that moving targets are a thing, but this one is moving too fast for me.
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on June 01, 2021, 12:44:25 pm
Today we have a working VPN
We want to discard it to go through the LAN and what we are discussing.

So VPN is out of the scope
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on June 02, 2021, 10:01:14 am
Please confirm that you tried to ping between 192.168.2.111 and 192.168.2.112
YES

Which rules did you create on which OPNsense, to allow the ICMP packages?
allow all from any on LAN and lan2lan

When you enable logging for the default drop policy in OPNsense, do you see the ICMP packages being dropped?
nope

There should be a checkbox somewhere in system settings / logging.
yes
Title: Re: 2 LAN …. No communication
Post by: stanthewizzard on June 07, 2021, 11:49:41 am
SOLVED

[OPNsense 1 Port 1]-----[Switch 1]-----LAN 1 clients
[OPNsense 1 Port 2]
                 |
                 |
             [Switch 0]
                 |
                 |
[OPNsense 2 Port 2]
[OPNsense 2 Port 1]-----[Switch 2]-----LAN 2 clients

[OPNsense 1 Port 1] - 192.168.0.1
[OPNsense 1 Port 2] - 192.168.1.10

[OPNsense 2 Port 2] - 192.168.1.1
[OPNsense 2 Port 1] - 192.168.0.10

[OPNsense 1 gateway1] - 192.168.0.1
[OPNsense 1 gateway2] - 192.168.1.1

[OPNsense  gateway1] - 192.168.1.1
[OPNsense  gateway2] - 192.168.0.1

[OPNsense 1 route1] - 192.168.1.0/24 -> GW192.168.1.1
[OPNsense 2 route1] - 192.168.0.0/24 -> GW192.168.0.1

It works :)
Title: Re: [SOLVED] 2 LAN …. No communication
Post by: Sheldon on June 23, 2021, 12:37:15 pm
I didn't receive a notification mail about your recent post, but i saw it just now. I'm glad that you finally found a solution and posted it here :-)