Adding an additional LAN interface

[jbresee]

Hey -
I'm new to Opnsense, coming from a Fortinet background.
I'm adding a new optical interface for a lan switch. I've got it set up with DHCP dishing out addresses as expected.
I've tried to add firewall rules to allow pinging the firewall, and a rule to allow all outbound access.
I done a packet capture on the interface as I ping the firewall. I see the inbound ping, but no response.
I suspect I've done something stupid with my rules.
Can you guys take a peak and see if you see something obvious?

[EricPerl]

Any clue in the FW live view?
New IP range not overlapping with another, right?

[bartjsmit]

DHCP sits between layer 2 (network) and layer 3 (IP). You can't have more than one DHCP server per (broadcast) network. Additionally when you add an interface to OPNsense (e.g. OPT1), it is seen as a separate network and no default firewall rules are created for it.

You essentially have three options:
- Leave your networking to your switch infrastructure and only have one OPNsense LAN interface
- Separate your network into security zones, each with their own IP subnet and DHCP (what @EricPerl hinted at)
- Create a bridge on OPNsense for your LAN interfaces

Bart...

[jbresee]

Any clue in the FW live view?
New IP range not overlapping with another, right?

current lan segment is 192.168.2.0/24
new segment is 192.168.4.0/24

interesting - live view shows dns queries matching on the allow everything outbound rule.

[jbresee]

DHCP sits between layer 2 (network) and layer 3 (IP). You can't have more than one DHCP server per (broadcast) network. Additionally when you add an interface to OPNsense (e.g. OPT1), it is seen as a separate network and no default firewall rules are created for it.

You essentially have three options:
- Leave your networking to your switch infrastructure and only have one OPNsense LAN interface
- Separate your network into security zones, each with their own IP subnet and DHCP (what @EricPerl hinted at)
- Create a bridge on OPNsense for your LAN interfaces

Bart...

Thanks for the reply Bart!
I'm trying to do option 2 above - I have separate address ranges for each interface, and dhcp is correctly handing out addresses on each segment.
I have added the following default rules to the new optical interface:
- allow ping to the firewall addresses
- allow any outbound traffic to the wan interface

Watching liveview, I can see pings are getting to the ICMP rule, but not receiving a reply. I also see DNS queries hitting the outbound traffic rule and passing, but no reply traffic.

I have NAT set to hybrid.

I'm not sure what I've got wrong here!  Thanks again for the help!

[EricPerl]

A few comments.

The rule you show in your original post is pretty much allow all in (from the perspective of the firewall) on that interface, only restricting the source to the entire subnet (which is not much of a restriction in common cases). It allows more than ICMP echo...

The firewall logs only ever show the "initial" traffic.
For a HTTPS connection to an external website, with logging fully enabled (per rule and per advanced FW settings), you'd see an in on SFP1, then out on WAN (after NAT). All subsequent traffic (until the connection is closed) is not visible there.

For ICMP, if you just ping SFP1.Address, you just see 1 in on SFP1 (the reply and subsequent request/reply don't show).
In this case, because there's no connection to be closed, state is kept until it times out after a short period of inactivity.
If you ping an internet address, you'd see 1 out on WAN (after NAT).

For DNS, assuming you're using a DNS server on OPN, you'd see 1 in from LAN or SFP1 (depending where the client lives).
Whether you see something out on WAN depends whether the answer is cached... This is not a pure routing use case.
DNS over UDP uses different source ports for every request so you see a lot of these...

If you want to see all traffic, you need to resort to packet captures in interfaces > diagnostics.

[jbresee]

Hey Eric,

Thanks for the info. I was unaware of how the logging was working, so that's helpful. I will do some packet captures to get a better picture.
The intent of the rule was to allow all traffic outbound on the new lan interface (SFP1), and it looks like it's doing that.

I still haven't figured out why traffic appears to make it to the rules, pass the outbound rules, but nothing returns.

[EricPerl]

Just from a terminology perspective, the FW is using direction in and out from its perspective.
That's what you have in your rule ;-)
While outbound can mean to the internet (or another interface) from the perspective of an interface, it can get confusing.
Outbound is a little weird when pinging the GW of the interface you're on...

That "pass all" rule should clearly be sufficient to allow pinging the GW. No need for another rule.
You apparently see an entry in the FW log. That's a start.

For the packet capture, select the interface, specify IPv4, ICMP and the client as host.
You should get something like this (high detail):
You cannot view this attachment.

[jbresee]

I did a package capture and I see all the traffic originating on from my test system. I don't see any traffic headed out from the firewall to the test system.

I'm missing something obvious here, but I can't figure out what it is!

Also, I've deleted the extra rules allowing ping, since the allow all from the lan covers ping.

[EricPerl]

That should be such a simple test...

What kind of NIC are we talking about?

Do you mind sharing a screenshot of your interfaces > overview screen?
Feel free to mask your public IPs.

[jbresee]

The interface is an SFP port.
Since I am getting a DHCP address on the client, and I can see traffic coming from the client to the firewall, and in live view I can see the packet arriving at the firewall and passing the allow all out rule, it seems like basic connectivity is up.

I've attached a screen shot for both the existing LAN interface and the new SFP interface.

I also attached a packet capture from the firewall showing traffic coming from the client machine with a ping that is not responded to.
Thanks for any thoughts!

[Gautier]

Stupid answer but reboot. There is some config works only on reboot.

[EricPerl]

I was looking for brand/model. Some Realtek NICs have a bad rep, some fixable with a driver plug-in.
I don't remember what the symptoms are.

Did you do that packet capture directly on the client?
It doesn't show any replies (DNS, MDNS, SSDP) but that could be due to filtering.

I suggested a capture on OPN (Interfaces > Diagnostics). Trying to establish is packets are sent.