3rd Interface Not Working as Expected

[Pfirepfox]

Hi All,

I have been using OPNSense for ages in a pretty basic manner with just two interfaces being assigned (WAN & LAN) and a handful of services to up the security (Unbound, IPS, GeoBlocking, and some ACLs), with everything working great. However, recently I added a few security cameras to my house and for security wanted to place this on a new physical interface of my OPNSense device rather than it being a VLAN (My device has 4 ports so it seemed better to just create a new interface rather than mess around with VLANs).

The new interface and network has been assigned and it is a direct replica of my LAN, just with a different subnet. I configured some basic rules such as the below:

• No WAN traffic on camera network
• Camera network to camera network allow all (For the NVR + NAS)
• Camera network to LAN allow all (So the cameras and NVR are accessible by other devices)
• LAN to Camera network allow all (as per above)

NAT is default with nothing special in it.

This is where it gets a bit weird, everything works as expected for around 12 to 24 hours but then everything on the Camera network becomes unavailable with no apparent reason (to me anyway). I have changed the down stream switch (just for the Camera Network), all ethernet cables, and also tried the 4th port on my OPNSense device, however, the issue persists.

I have also looked at IPS, local DHCP, Firewall rules, and Unbound but havent found any logs which may indicate something is incorrect, as on boot everything works correctly. Restarting the network devices brings everything backup without an issue.

At this stage i have exhausted what i can think of, does anyone else have an idea of what it could be?

[Pfirepfox]

I have been investigating further and still cannot find an issue with my configuration. I have essentially followed the Setup Guest Guide here: https://docs.opnsense.org/manual/how-tos/guestnet.html#step-1-configure-interface, but rather than block rules added allow rules.

I can see the appropraite firewall rule is being hit on the LAN side but no response from the camera network, I'm really not sure what is going on here as everything works initially and then drops after a period of time. I think it is something to do with DHCP as whenever it drops i dont see an active lease for the camera network (192.168.51.1/24)

[Pfirepfox]

I have narrowed this down to the DHCP service, as i migrated the entire setup under my standard LAN network and it also dropped after 12 -24 hours even though all other devices on the LAN network remained unaffected. When i checked on the status no DHCP lease was active for the security cameras and NCR even though i configured them as static entries.

Can anyone think of why DHCP for certain devices is not working while other devices on the same network are? The only common denominator is the PoE switch which powers the devices, however, i have already replaced it once before. It is the same type though so i wonder if this type has an issue, or maybe the batch? But that seems unlikely...

[cookiemonster]

• Camera network to camera network allow all (For the NVR + NAS)

This rule isn't needed right? Devices on the same network talk to each other via the switch without going through the firewall.
But not the reason for what's happening.
Switch on some power-saving setting that makes the interface go down according to OPN, maybe?
Anything in dmesg ?

[Pfirepfox]

Thanks for the reply i am really stuck here, sorry for the late reply, i have been on a work trip.

Since getting back i have reset OPNsense to a bare bones installation (Reinstall) and configured only the most basic settings via the terminal setup (WAN, LAN, and my Camera Network). I ported over the most critical settings being the static DHCP settings to ensure all the connections are still valid, however, it dropped again :/

Correct the rule does not need to be there but i was lost as why it was not working so was trying everything, in the bare bones setup i have now, i have no firewall rules, IPS/IDS, etc which can interfere yet it still occurs.

I dont believe i had any power options enabled, as my reinstall would have removed the ones i had anyway.

For dmesg what are you referring to here?

On the firewall logs i can see my traffic is allowed to the Camera network, but i never see anything else return. I must admit i am pretty perplexed by this, as i could imagine i had a weird setting turned on i forgot about initially, but the fact it continue to drop after the most basic setup is crazy. I have also gone through 3 downstream switches and several cables to try and resolve this, i think the bug is with Opnsense itself?

[cookiemonster]

lost track.
Show the interface setup and firewall rules for it please.

[Pfirepfox]

Please see the attached. Everything is default apart from the two firewall rules allowing LAN --> Camera and Camera --> LAN

[cookiemonster]

Thanks for this. Rule 3 is covered by rule 1 so it is surplus. Not a problem though.
The rules otherwise and the setup don't have a reason for the traffic to not return.
All I can think of is the interface detaching. Physically loose connection or logical ie. a PoE port that goes to sleep.
You'll need to hunt around for clues in logs. And when it happens, go to console and check ifconfig for it showing UP.

[Pfirepfox]

Yeah that makes sense regarding the firewall rules.
I don't think it is physical as I have tried other ports on the Opnsense device (I have two free) as well as many cables, I have also replaced the poe switch, all with the same result.

Each time it happens it is only the camera network that goes down with the LAN behaving as normal. It is like the DHCP request are not being served after a period of time as each time I check the leases are expired. I have check all the logs I can think of but nothing appears to be the cause from what I can find, I am sure it is in there though.

Does opnsense have a sleep function or something?

[cookiemonster]

No it does not have a sleep function.

[Greg_E]

Do you have a spare computer that you can use to set up a DHCP server for that LAN? I'm partial to Zentyal community edition, but you could use a Windows server eval or any other OS you are comfortable using. Then you could turn off the DHCP offered by OPNsense and see if there is still a problem.

if not you may need to set up Wireshark and record that network to see when the DHCP requests are coming in, and what gets answered back, this should all be broadcast traffic. If not, you might need to mirror the port for the OPNsense connection, I think you can do this inside of OPNsense, but not sure if I'm remembering correctly.

[cookiemonster]

Problem seems to be not with the traffic but with it stopping after a while. Packet capture would only confirm traffic stops, no?
The only reason I can think of traffic stopping after a while is hardware problem or setting i.e. PoE or some other power saving feature.

[Pfirepfox]

DHCP is configure using the default opnsense settings, I have set static IPs but I can't imagine that those would cause any issues. DHCP work fine on the LAN but on Camera it appears to be the issue. I haven't changed any setting apart from enabled DHCP on the Caerma network as well as the static IPs.

I don't think it is hardware as I have stated before I have rotated ports, cables, as well as the PoE switch.

[cookiemonster]

Static leases outside the dhcp pool, right?

[Pfirepfox]

Correct, DHCP pool starts at 192.168.51.20 --> 192.168.51.60, with all static leases within 192.168.51.2 --> 192.168.51.6