OPNsense Forum

English Forums => 25.1, 25.4 Production Series => Topic started by: ikkeT on April 08, 2025, 08:10:32 AM

Title: WHY ARP has ip bouncing from device to another
Post by: ikkeT on April 08, 2025, 08:10:32 AM
Hi,

I have had some unstableness in my opnsense for over a year now. After long digging, I found it is likely caused by ARP jumping IP from device to another in my laptop. Why does this keep happening?

I have laptop with two interfaces, wlan and usbdongle ethernet when in wire:

2: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6a:cb:3f:c6:c9:09 brd ff:ff:ff:ff:ff:ff permaddr 9c:67:d6:0f:8f:c0
    inet 192.168.117.59/24 brd 192.168.117.255 scope global dynamic noprefixroute wlp0s20f3
       valid_lft 4000sec preferred_lft 4000sec
    inet6 fe80::66f9:af89:6d28:703a/64 scope link tentative noprefixroute
       valid_lft forever preferred_lft forever
4: enp0s13f0u1u2u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 48:65:ee:15:7f:c2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.117.56/24 brd 192.168.117.255 scope global dynamic noprefixroute enp0s13f0u1u2u1
       valid_lft 2936sec preferred_lft 2936sec
    inet6 fe80::4993:59f0:25d:240a/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

Both MACs are fixed with separate IP addresses in KEA reservations page:

192.168.117.0/24   192.168.117.56   48:65:ee:15:7f:c2   satechi
192.168.117.0/24   192.168.117.59   6a:cb:3f:c6:c9:09   iklap

Satechi is the usbdongle brand. Iklap is the Fedora laptop name.

While I have the both connected (docking), I see this bouncing in OPNSense:

arp: 192.168.117.56 moved from 6a:cb:3f:c6:c9:09 to 48:65:ee:15:7f:c2 on igb2
arp: 192.168.117.59 moved from 6a:cb:3f:c6:c9:09 to 48:65:ee:15:7f:c2 on igb2
arp: 192.168.117.56 moved from 48:65:ee:15:7f:c2 to 6a:cb:3f:c6:c9:09 on igb2
arp: 192.168.117.56 moved from 6a:cb:3f:c6:c9:09 to 48:65:ee:15:7f:c2 on igb2
arp: 192.168.117.56 moved from 6a:cb:3f:c6:c9:09 to 48:65:ee:15:7f:c2 on igb2

And I believe that will drain the opnsense out of mem soonish. What causes the IP to bounce outside of their mac? I suspect it's somehow the laptop sending dhcpc query with laptop name in it (NetworkManager), which then KEA uses to overrule what Reservations page is saying.

Is this a bug somewhere? Why does KEA allow ip to go from MAC to another not respecting reservations?

Any idea what should be done here? It's annoying needing to toggle wlan off each time while docking due this. Do I have some misconfig in a) in my Fedora laptop or b) KEA, or c) bug somewhere?

Title: Re: WHY ARP has ip bouncing from device to another
Post by: Patrick M. Hausen on April 08, 2025, 09:32:46 AM
Services > Kea > Subnets > edit your subnet and remove that check mark:

(https://forum.opnsense.org/index.php?action=dlattach;attach=43961;image)
Title: Re: WHY ARP has ip bouncing from device to another
Post by: ikkeT on April 08, 2025, 10:18:46 AM
thanks, makes sense now that you point it out :D
Title: Re: WHY ARP has ip bouncing from device to another
Post by: chemlud on April 08, 2025, 11:02:27 AM
Quote from: Patrick M. Hausen on April 08, 2025, 09:32:46 AMServices > Kea > Subnets > edit your subnet and remove that check mark:

(https://forum.opnsense.org/index.php?action=dlattach;attach=43961;image)

...eeehm, this was introduced when?
Title: Re: WHY ARP has ip bouncing from device to another
Post by: Patrick M. Hausen on April 08, 2025, 11:20:43 AM
Quote from: chemlud on April 08, 2025, 11:02:27 AM...eeehm, this was introduced when?

To my knowledge when Kea was first added to OPNsense. Client-ID instead of MAC is the default behaviour for Kea.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: chemlud on April 08, 2025, 11:47:18 AM
Ah,, sorry, forgot my Alzheimers medis this morning... ISC was the old *sense standard, KEA was introduced as possible replacement.

What is the current status on DHCP in the roadmap of OPNsense? Will ISC persist? I think I read about something going to be blown away soonish...
Title: Re: WHY ARP has ip bouncing from device to another
Post by: Patrick M. Hausen on April 08, 2025, 11:58:52 AM
Quote from: chemlud on April 08, 2025, 11:47:18 AMWhat is the current status on DHCP in the roadmap of OPNsense? Will ISC persist? I think I read about something going to be blown away soonish...

ISC is going to be removed.
DNSmasq DHCP is the recommended replacement for small installations.
Kea is the recommended replacement for enterprise environments (once it reaches feature parity).

I don't know about the timeline from the top of my head.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: ikkeT on April 08, 2025, 12:01:47 PM
Damn, then I made the migration from dnsmasq to kea for nothing, I thought kea was the way forward. Well it works now...
Title: Re: WHY ARP has ip bouncing from device to another
Post by: franco on April 08, 2025, 12:31:03 PM
> Damn, then I made the migration from dnsmasq to kea for nothing

But not on our end? Dnsmasq DHCP is coming in 25.1.6 so if you already have moved your DHCP to OPNsense you just need to reconfigure from there.


Cheers,
Franco
Title: Re: WHY ARP has ip bouncing from device to another
Post by: ikkeT on April 08, 2025, 12:54:03 PM
I had dnsmasq in use in opnsense, and moved the config to kea and unbound. It's no biggie just one evening useless work. I don't need to do anything to keep the current setup as it sounds.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: DEC670airp414user on April 08, 2025, 01:03:20 PM
interesting.  I've noticed weirdness since I've used kea.

i turned this setting off. we will see if that clears up some it

"ISC is going to be removed.
DNSmasq DHCP is the recommended replacement for small installations."

i wish i had read/ known that before i moved to kea...
Title: Re: WHY ARP has ip bouncing from device to another
Post by: alex303 on April 08, 2025, 01:31:16 PM
Quote from: Patrick M. Hausen on April 08, 2025, 09:32:46 AMServices > Kea > Subnets > edit your subnet and remove that check mark:

(https://forum.opnsense.org/index.php?action=dlattach;attach=43961;image)

Sry to hijack this thread but, are these client identifiers unique for each device? Can this feature of kea enhance security in a sense that if someone spoofs the mac address of device, he still wont be able to connect and receive ip address from dhcp server because the client identifier doesnt match ? Assuming of course that dhcp static mapping is in full use.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: franco on April 08, 2025, 01:34:51 PM
> i wish i had read/ known that before i moved to kea...

I think we are all on the same side here.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: Patrick M. Hausen on April 08, 2025, 01:36:47 PM
Quote from: franco on April 08, 2025, 01:34:51 PMI think we are all on the same side here.

Then why introduce YAS (yet another service)? What can DNSmasq do that Kea cannot?
Why do we need anything but Kea and BIND? Special filtering services like AGH notwithstanding, of course.
Title: Re: WHY ARP has ip bouncing from device to another
Post by: franco on April 08, 2025, 02:11:23 PM
> What can DNSmasq do that Kea cannot?

It's much more reliable and straightforward and compact. Dnsmasq can do DNS, DHCPv4/6 and RA (and register your DHCP leases in DNS). Everything is already implemented without too much effort in OPNsense development release. The only thing it cannot do is prefix delegation and that's why there will be Kea v6 eventually.

To outline this a bit more for 26.1 we will have a compact default setup using DHCP via Dnsmasq or an optional complicated one using Kea/radvd for all things HA and of grander scale.


Cheers,
Franco
Title: Re: WHY ARP has ip bouncing from device to another
Post by: Patrick M. Hausen on April 08, 2025, 02:32:00 PM
Quote from: franco on April 08, 2025, 02:11:23 PMTo outline this a bit more for 26.1 we will have a compact default setup using DHCP via Dnsmasq

Then why not also DNS via DNSmasq? Doesn't make sense to me. Also the reliability claim comes as a big surprise to me. But that's better discussed face to face over a beer ;-) If at all.

My point is: you are introducing dependencies on "a gazillion" external projects most of which have no strong ties to BSD and might at one point not even compile without patches on FreeBSD. That's a potentially huge integration workload so there better be a good reason for that.

Similarly why radvd - according to their own documentation a "Linux IPv6 Router Advertisement Daemon" - when rtadvd is built in right into FreeBSD?
Title: Re: WHY ARP has ip bouncing from device to another
Post by: franco on April 08, 2025, 02:39:15 PM
> Then why not also DNS via DNSmasq? Doesn't make sense to me. Also the reliability claim comes as a big surprise to me. But that's better discussed face to face over a beer ;-) If at all.

The blocklist in Unbound and other features make it more desirable than Dnsmasq for DNS, even for the defaults. Besides, *sense switched away from Dnsmasq DNS a long time ago for same or similar reasons that are still valid today.

> My point is: you are introducing dependencies on "a gazillion" external projects most of which have no strong ties to BSD

Dnsmasq was already in there. Kea is unavoidable. I think this is the problem with modern DHCP.  :)

> Similarly why radvd - according to their own documentation a "Linux IPv6 Router Advertisement Daemon" - when rtadvd is built in right into FreeBSD?

If it's already in there why not. It's been proven to work. They merged our code for FreeBSD integration in 2.20. ;)


Cheers,
Franco