Tasmota (ESP) devices not obtaining lease since 22.1.8

[Grossartig]

I have a bunch (like 20 or so) ESP devices at home that are flashed with the Tasmota firmware (https://tasmota.github.io/docs/). Since OPNsense 22.1.8, they seem to be unable to obtain a lease from my OPNsense DHCPv4 service.

I tried adding a new Tasmota device to my network (to rule out any issues with the existing devices), and it, too, cannot obtain a lease from DHCPD.

The DHCPD logs in OPNsense do not show me anything of value, and I am unable to obtain any logs from the ESP devices, as I cannot access them after trying to add them to my network.

I have a WiFi AP sitting behind my OPNsense box that all WiFi devices connect to, including my ESP/Tasmota devices. This AP works fine, everything connects without issues. I also rebooted it for good measure. I don't believe the WiFi AP is the issue, but obviously cannot rule it out completely.

My wife is furious, as the lights in our apartment don't work right now (on account of them all being Tasmota based devices)

[franco]

Did you try reverting to 22.1.7 yet?

Change that may be interfering is https://github.com/opnsense/core/commit/f527ee2d198


Cheers,
Franco

[Grossartig]

Not yet -- like this, right?

Code: [Select]

opnsense-revert -r 22.1.7 dhcpd

[franco]

Almost, I suspect core package... there are no dhcpd binary changes I think

# opnsense-revert -r 22.1.7 opnsense


Cheers,
Franco

[Grossartig]

Thanks Franco -- I had high hopes but reverting to 22.1.7 didn't alleviate this issue. My existing (old) ESP/Tasmota devices are still seemingly not being given a lease, and a new Tasmota device I am trying to add also doesn't get an IP address.

I also disabled my CARP backup node (which is still on 22.1.8 ), just to be sure there's no interference from it, and I also rebooted my WiFi AP to ensure all devices would be forced to reconnect and request an IP address.

Perhaps unrelated to 22.1.8? What's perhaps also interesting is that all these ESP devices (bulbs & wall switches of different brands) lost connectivity while I was asleep in the middle of the night. Not one by one, but seemingly all at the same time.

[franco]

Could be coincidental but I'm not sure. As of late I had issues with a unifi AP behaving strangely weird dropping out after 1-2 minutes after firewall boot. Turned out I had created a virtual IP and a proxy ARP binding for testing purposes all mapped to the same IP and that prevented DHCP from working, but only after a day of holding the configuration... but that's likely not what happened here. Just to illustrate what can go wrong.


Cheers,
Franco

[Grossartig]

Would it be worthwhile reverting the kernel as well? The previous version was 22.1.5, correct? So like this?

# opnsense-revert -r 22.1.5 kernel

Edit: No, like this:

# opnsense-update -kr 22.1.5

Update: Reverting kernel didn't solve it either. Still on the hunt...

[Grossartig]

I'm at my wits end and have been looking into this for hours now. When I daisy-chain another WiFi router (with its own DHCP server) behind my WiFi-AP via Ethernet (which itself is connected to the OPNsense LAN port), my Tasmota devices are able to connect to it and obtain an IP address from it. But when OPNsense is the DHCP server, my Tasmota devices cannot get a lease (although other devices have no issues).

I made no configuration changes in the past two days, expect for updating from OPNsense 21.1.7 to 22.1.8. But downgrading core and kernel has to effect.

Stumped.

[meyergru]

Maybe a dumb question to ask, but since I had a similar problem with only some devices getting no lease and also because I have a Tasmota device that gets DHCPv4 leases just fine with 22.1.8_1:

You did not set the DHCP response delay by any chance? See: https://forum.opnsense.org/index.php?topic=22047

[Grossartig]

Thanks for the response! The response delay field is empty on my end.

Also, I found out that my Tasmota devices connect just fine and can obtain a lease when connected through a different (yet identical) WiFi AP that is connected to a different OPNsense port (OPT1). Did a lot of testing with the different APs and narrowed the issue down to the OPNsense LAN port -- any Tasmota devices that are behind that do not get a lease. Still didn't figure out why yet, but if I put the Tasmota devices behind OPT1, everything is fine. And this only started last night.

[Grossartig]

Ah dang, I now have other devices that are unable to receive a lease from OPNsense when behind the LAN port. In this new case, it's an Android TV device.

[meyergru]

Looks like no devices get any new leases and they drop out after the old lease has expired. Can you see incoming DHCP requests in your logs?

Are both OTP1 and LAN bridged or in different DHCP zones? Maybe the available DHCP range of one zone is exhausted?

Also, some switches have the ability to block DHCP answers from non-privileged ports.

[Grossartig]

Thanks man for bouncing some ideas off with me here, I appreciate it!

Here's more insight after more testing:

1) It's not affecting all devices. I was able to connect with a Windows laptop that had never connected to my network before, and that worked fine. However, my Tasmota devices and my Android TV are a no-go.

2) Regarding the Android TV, when I try to connect to my network, I don't see anything in the dhcpd/latest.log, nor in the system/latest.log when I try to connect. I wonder which other logs I should look at?

3) LAN and OPT1 are in different DHCP zones. The first for my regular LAN stuff, the second for guest/IoT kind of access. I double checked that the LAN IP range is definitely not exhausted yet.

All devices I've had issues with connecting to LAN I am successfully able to connect to my network behind OPT1.

[meyergru]

If your OpnSense does not see the DHCP requests (which I guess from nothing appearing in your DHCP logs), it must be either a firewall problem on LAN or something before that on your switch that blocks certain request from arriving at your OpnSense.

What is strange is that this affects only some devices...

[defaultuserfoo]

Check /var/dhcpd/etc/dhcpd.conf to see if there is something in there that's weird.

Disable and re-enable the dhcp service.

Restart the dhcp service.

Restart the router.

Run a packet capture on the LAN interface and see what's going on with DHCP packages (Interfaces-->Diagnostics).

Make sure there is no other dhcp server somewhere on the network.

Replace the network cable going to that access point, maybe other involved cables as well.  (Yes, seriously; even network cables can be weird.)

Reset the access point to default settings and reconfigure it from scratch.