Device Monitor - a tool for automatic network device monitoring and detection

Started by hacesoft, January 02, 2026, 01:01:40 PM

Previous topic - Next topic
Quote from: hacesoft on April 21, 2026, 06:51:56 PMNot every plugin is for everyone — install what fits your needs. 🙂
I didn't mean to belittle your effort. In fact, I appreciate every effort to improve OPNsense.

I just not sure what the size of the target audience for all this host discovery stuff is, yours and the new component of OPNsense. If I want to know what's going on on my network, I ask it directly via SNMP or an all-in-one solution like Unifi or Omada. Only small, unmanaged networks don't already provide that functionality. How many are there using OPNsense?

I have an older version of Device Monitor installed and working.  It is great !!!

I am happy with it, but is there a way to update to this new version, or do I need to fully uninstall/reinstall?

Thank you.

Quote from: hacesoft on April 21, 2026, 06:38:33 PMHi Steffen,
The hostname field works as follows: the plugin pulls the hostname from Services → ISC DHCPv4 / DHCPv6 → DHCP Static Mappings — specifically the Hostname field for each entry. If a device doesn't have a static mapping with a hostname defined there, the field will simply remain empty, as the plugin has no other automatic source for this information.
You can also fill in the hostname manually directly in the plugin, but note that this is stored only in the plugin's own database — it does not propagate back to OPNsense DHCP or any other system.
So the short answer: populate the Hostname field in your DHCP static mappings and it will appear automatically.

I have hostnames defined at Dnsmasq because I think ISC is deprecated. So it would be nice to support Dnsmasq too.

Quote from: pc44 on April 23, 2026, 04:26:58 AMI have an older version of Device Monitor installed and working.  It is great !!!

I am happy with it, but is there a way to update to this new version, or do I need to fully uninstall/reinstall?

Thank you.

Figured it out.  Deleted the existing /tmp/opnsense-devicemonitor folder.  Then downloaded, unzipped, and copied over the new files.  Then just re-ran sh install.sh.

Now up-to-date. ☑️

Quote from: pc44 on April 23, 2026, 09:42:44 PM
Quote from: pc44 on April 23, 2026, 04:26:58 AMI have an older version of Device Monitor installed and working.  It is great !!!

I am happy with it, but is there a way to update to this new version, or do I need to fully uninstall/reinstall?

Thank you.

Figured it out.  Deleted the existing /tmp/opnsense-devicemonitor folder.  Then downloaded, unzipped, and copied over the new files.  Then just re-ran sh install.sh.

Now up-to-date. ☑️
Good day, exactly as you asked. If you use sh install.sh, everything except the database will be deleted before installation. After installation, the plugin will be started again. If you use sh uninstall.sh, everything will be removed.

Quote from: SteffenDE on April 23, 2026, 07:49:14 AM
Quote from: hacesoft on April 21, 2026, 06:38:33 PMHi Steffen,
The hostname field works as follows: the plugin pulls the hostname from Services → ISC DHCPv4 / DHCPv6 → DHCP Static Mappings — specifically the Hostname field for each entry. If a device doesn't have a static mapping with a hostname defined there, the field will simply remain empty, as the plugin has no other automatic source for this information.
You can also fill in the hostname manually directly in the plugin, but note that this is stored only in the plugin's own database — it does not propagate back to OPNsense DHCP or any other system.
So the short answer: populate the Hostname field in your DHCP static mappings and it will appear automatically.

I have hostnames defined at Dnsmasq because I think ISC is deprecated. So it would be nice to support Dnsmasq too.
Good day, I just converted my home network from ISC to DNSmasq and it will take a while before I get to it and modify the plugin. but don't worry, it will happen :).

Quote from: mooh on April 21, 2026, 10:01:40 PM
Quote from: hacesoft on April 21, 2026, 06:51:56 PMNot every plugin is for everyone — install what fits your needs. 🙂
I didn't mean to belittle your effort. In fact, I appreciate every effort to improve OPNsense.

I just not sure what the size of the target audience for all this host discovery stuff is, yours and the new component of OPNsense. If I want to know what's going on on my network, I ask it directly via SNMP or an all-in-one solution like Unifi or Omada. Only small, unmanaged networks don't already provide that functionality. How many are there using OPNsense?

Hello, I recently switched from pfSense to OpnSense and this option was standard in the system and I use it to check if an uninvited guest is connecting. Or when I connect a device, I find out its IP address. I don't have unifi, or rather I only use AP AC RL and I have a classic L2, L3 switch. So OpnSense will provide me with the necessary information :). I originally wrote the plugin just for myself, but I decided to share it with others... I would like the plugin to be offered as a standard package, but that's too much for me for now :).

Good day, I have released version 2.1. which supports Dnsmasq DNS & DHCP.
Again I look forward to your reactions :).

The link is:

https://github.com/hacesoft/opnsense-devicemonitor

Quote from: hacesoft on April 25, 2026, 08:09:40 PMGood day, I have released version 2.1. which supports Dnsmasq DNS & DHCP.
Again I look forward to your reactions :).

The link is:

https://github.com/hacesoft/opnsense-devicemonitor

Thank you for this, that's better - my Hostnames are now correct.

Hi,

I'm running OPNsense 26.7.2_2 with Device Monitor v2.1.

I found an issue where multiple historical Hostwatch records for the same MAC address can cause Device Monitor to select an old IP/interface record instead of the current one.

Example

My Windows PC has:

MAC: 7c:83:34:b2:34:7c
Current IP: 192.168.20.111
Interface: igc1 / LAN

Device Monitor initially showed this device incorrectly as:

IP: 192.168.20.146
Interface: IGC1
Offline

I checked the Hostwatch database directly:

sqlite3 /var/db/hostwatch/hosts.db \
"SELECT interface_name, ip_address, ether_address, first_seen, last_seen, organization_name
FROM v_hosts
WHERE ether_address='7c:83:34:b2:34:7c';"

Hostwatch contained three records for the same MAC:

igc1  192.168.20.146  2026-08-15 03:04:46
igc0  192.168.20.146  2026-08-15 11:10:03
igc1  192.168.20.111  2026-08-19 06:06:49

The last record is the current/correct one.

I also confirmed Hostwatch itself was updating correctly. For example, its most recent records included:

7c:83:34:b2:34:7c  192.168.20.111  igc1  2026-08-19 06:08:51

So Hostwatch was not the problem.

Cause

I traced this to:

/usr/local/opnsense/scripts/OPNsense/DeviceMonitor/scan_network.py

The Hostwatch query currently retrieves all matching records:

SELECT
    interface_name,
    ip_address,
    ether_address,
    first_seen,
    last_seen,
    organization_name
FROM v_hosts
WHERE protocol = 'inet'
  AND ether_address NOT IN ('ff:ff:ff:ff:ff:ff', '00:00:00:00:00:00')
  AND ip_address NOT LIKE '169.254.%'
ORDER BY last_seen DESC

There is no deduplication by ether_address.

Device Monitor's own database uses the MAC address as the key:

mac TEXT PRIMARY KEY

Therefore, when multiple Hostwatch records exist for the same MAC, they are processed sequentially and can overwrite one another. In my case, the historical .146 record ended up being stored instead of the current .111 record.

This also affected the Online/Offline status because Device Monitor was evaluating the stale record's last_seen.

Test fix

I tested a local modification to the SQL query which selects only the newest last_seen record for each MAC address, with the Hostwatch id used as a tie-breaker.

After applying the change and running a full scan, Device Monitor immediately changed the affected PC to:

MAC:      7c:83:34:b2:34:7c
IP:        192.168.20.111
Interface: IGC1
Status:    ONLINE

The Device Monitor database then showed:

7c:83:34:b2:34:7c
192.168.20.111
IGC1
2026-08-19 06:18:58
is_active = 1

I subsequently restarted Device Monitor and confirmed the service restarted normally and the Devices page continues to show the correct IP/interface/status.

Suggested fix

I believe the Hostwatch query should deduplicate records by ether_address and select the record with the newest last_seen before passing the results to the Device Monitor database.

This would prevent historical Hostwatch records from overwriting the current record for the same MAC.

The email notification functionality otherwise works correctly for me. I have successfully tested detection of a new MAC address and received the Device Monitor email notification through Postfix.

Thanks — I've attached modified SQL/query if useful.

Hi apg1959,

thank you for the very detailed report and for taking the time to trace this down to the Hostwatch records.

I checked the Device Monitor code and can confirm that your analysis is correct.

The problem is caused by the combination of:

```sql id="qq5f10"
ORDER BY last_seen DESC
```

in the Hostwatch query and the fact that Device Monitor subsequently processes every returned record.

Since the Device Monitor database uses:

```sql id="i1hxkw"
mac TEXT PRIMARY KEY
```

multiple Hostwatch records belonging to the same MAC are processed sequentially. The newest record is processed first, but an older record for the same MAC can then overwrite its IP address, interface and `is_active` state.

So in your example the processing can effectively become:

```text id="r11eop"
7c:83:34:b2:34:7c -> 192.168.20.111 (newest, correct)
7c:83:34:b2:34:7c -> 192.168.20.146 (older)
7c:83:34:b2:34:7c -> 192.168.20.146 (older)
```

and the historical record ends up in the Device Monitor database.

This also explains the incorrect Offline state, because `is_active` is recalculated using the `last_seen` value of each historical record.

### Proposed fix

I agree that Device Monitor should only pass the newest Hostwatch record for each MAC address to the rest of the scan logic.

One solution would be to do the deduplication directly in SQL, as you suggested.

However, I am currently considering doing the deduplication immediately after reading the Hostwatch result instead. Since the query is already sorted by:

```sql id="lh3gr4"
ORDER BY last_seen DESC
```

the first occurrence of each MAC is the newest one.

For example:

```python id="5ym4vr"
seen_macs = set()

for row in rows:
    mac = (row['ether_address'] or '').lower()

    if not mac or mac in seen_macs:
        continue

    seen_macs.add(mac)

    iface = row['interface_name'] or ''
    vlan = map_interface_to_vlan(iface)

    vendor = row['organization_name'] or 'Unknown'
    if len(vendor) > 40:
        vendor = vendor[:37] + '...'

    devices.append({
        'mac': mac,
        'ip': row['ip_address'] or '',
        'hostname': '',
        'vendor': vendor,
        'vlan': vlan,
        'first_seen': row['first_seen'] or '',
        'last_seen': row['last_seen'] or '',
    })
```

This keeps the Hostwatch SQL simple and also avoids depending on additional Hostwatch database fields for tie-breaking.

The result is that only the first (newest) record for a MAC is passed to Device Monitor, while all older historical records for the same MAC are ignored.

I would still be interested in seeing the exact SQL modification you tested, especially the tie-breaker using the Hostwatch ID. Please post it if you have it available. I can then compare both approaches before committing the fix.

Thanks again for the excellent report and for verifying the behavior before and after the change. Your testing made the cause very clear.

I will include a fix for this in the next Device Monitor update.

For future bug reports and feature requests, it would be even better to open an issue directly on the project's GitHub repository. It makes it much easier for me to track reported problems, discuss proposed changes, attach patches and link fixes or commits to the original report. The OPNsense forum is of course still useful for general discussion and user support, but GitHub Issues is the preferred place for reproducible bugs like this one.

Thanks!
         
Quote from: apg1959 on August 19, 2026, 09:01:35 AMHi,

I'm running OPNsense 26.7.2_2 with Device Monitor v2.1.

I found an issue where multiple historical Hostwatch records for the same MAC address can cause Device Monitor to select an old IP/interface record instead of the current one.

Example

My Windows PC has:

MAC: 7c:83:34:b2:34:7c
Current IP: 192.168.20.111
Interface: igc1 / LAN

Device Monitor initially showed this device incorrectly as:

IP: 192.168.20.146
Interface: IGC1
Offline

I checked the Hostwatch database directly:

sqlite3 /var/db/hostwatch/hosts.db \
"SELECT interface_name, ip_address, ether_address, first_seen, last_seen, organization_name
FROM v_hosts
WHERE ether_address='7c:83:34:b2:34:7c';"

Hostwatch contained three records for the same MAC:

igc1  192.168.20.146  2026-08-15 03:04:46
igc0  192.168.20.146  2026-08-15 11:10:03
igc1  192.168.20.111  2026-08-19 06:06:49

The last record is the current/correct one.

I also confirmed Hostwatch itself was updating correctly. For example, its most recent records included:

7c:83:34:b2:34:7c  192.168.20.111  igc1  2026-08-19 06:08:51

So Hostwatch was not the problem.

Cause

I traced this to:

/usr/local/opnsense/scripts/OPNsense/DeviceMonitor/scan_network.py

The Hostwatch query currently retrieves all matching records:

SELECT
    interface_name,
    ip_address,
    ether_address,
    first_seen,
    last_seen,
    organization_name
FROM v_hosts
WHERE protocol = 'inet'
  AND ether_address NOT IN ('ff:ff:ff:ff:ff:ff', '00:00:00:00:00:00')
  AND ip_address NOT LIKE '169.254.%'
ORDER BY last_seen DESC

There is no deduplication by ether_address.

Device Monitor's own database uses the MAC address as the key:

mac TEXT PRIMARY KEY

Therefore, when multiple Hostwatch records exist for the same MAC, they are processed sequentially and can overwrite one another. In my case, the historical .146 record ended up being stored instead of the current .111 record.

This also affected the Online/Offline status because Device Monitor was evaluating the stale record's last_seen.

Test fix

I tested a local modification to the SQL query which selects only the newest last_seen record for each MAC address, with the Hostwatch id used as a tie-breaker.

After applying the change and running a full scan, Device Monitor immediately changed the affected PC to:

MAC:      7c:83:34:b2:34:7c
IP:        192.168.20.111
Interface: IGC1
Status:    ONLINE

The Device Monitor database then showed:

7c:83:34:b2:34:7c
192.168.20.111
IGC1
2026-08-19 06:18:58
is_active = 1

I subsequently restarted Device Monitor and confirmed the service restarted normally and the Devices page continues to show the correct IP/interface/status.

Suggested fix

I believe the Hostwatch query should deduplicate records by ether_address and select the record with the newest last_seen before passing the results to the Device Monitor database.

This would prevent historical Hostwatch records from overwriting the current record for the same MAC.

The email notification functionality otherwise works correctly for me. I have successfully tested detection of a new MAC address and received the Device Monitor email notification through Postfix.

Thanks — I've attached modified SQL/query if useful.

New version released Device Monitor v2.3 : https://github.com/hacesoft/opnsense-devicemonitor

v2.3 (August 2026) — Direct SMTP and notification improvements

  • Added selectable Email delivery method in Device Monitor settings.
  • Local Sendmail / Postfix remains the default and preserves existing installations.
  • Added built-in Direct SMTP delivery using Python smtplib; no additional Python package is required.
  • Direct SMTP supports STARTTLS, SSL/TLS, and unencrypted SMTP.
  • Added SMTP server, port, username and password configuration to the UI.
  • Test Email now uses the currently selected delivery method.
  • Existing configuration files are merged with new defaults automatically, so upgrading does not require a configuration reset.
  • SMTP credentials are read from the protected Device Monitor configuration instead of being passed on the process command line.
  • Configuration containing SMTP credentials is stored with restrictive file permissions.
  • Includes the v2.2 Hostwatch fixes: newest Hostwatch record is selected per MAC, deleted devices no longer return from historical records, VLAN notification filtering was corrected, and real Hostwatch last_seen values are preserved during quick status updates.

Fantastic !!

Can't wait to update to this.  Will probably do so this evening.

Thank you for your work.