Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - hacesoft

#1
New version released Device Monitor v2.3 : https://github.com/hacesoft/opnsense-devicemonitor

v2.3 (August 2026) — Direct SMTP and notification improvements

  • Added selectable Email delivery method in Device Monitor settings.
  • Local Sendmail / Postfix remains the default and preserves existing installations.
  • Added built-in Direct SMTP delivery using Python smtplib; no additional Python package is required.
  • Direct SMTP supports STARTTLS, SSL/TLS, and unencrypted SMTP.
  • Added SMTP server, port, username and password configuration to the UI.
  • Test Email now uses the currently selected delivery method.
  • Existing configuration files are merged with new defaults automatically, so upgrading does not require a configuration reset.
  • SMTP credentials are read from the protected Device Monitor configuration instead of being passed on the process command line.
  • Configuration containing SMTP credentials is stored with restrictive file permissions.
  • Includes the v2.2 Hostwatch fixes: newest Hostwatch record is selected per MAC, deleted devices no longer return from historical records, VLAN notification filtering was corrected, and real Hostwatch last_seen values are preserved during quick status updates.
#2
Hi apg1959,

thank you for the very detailed report and for taking the time to trace this down to the Hostwatch records.

I checked the Device Monitor code and can confirm that your analysis is correct.

The problem is caused by the combination of:

```sql id="qq5f10"
ORDER BY last_seen DESC
```

in the Hostwatch query and the fact that Device Monitor subsequently processes every returned record.

Since the Device Monitor database uses:

```sql id="i1hxkw"
mac TEXT PRIMARY KEY
```

multiple Hostwatch records belonging to the same MAC are processed sequentially. The newest record is processed first, but an older record for the same MAC can then overwrite its IP address, interface and `is_active` state.

So in your example the processing can effectively become:

```text id="r11eop"
7c:83:34:b2:34:7c -> 192.168.20.111 (newest, correct)
7c:83:34:b2:34:7c -> 192.168.20.146 (older)
7c:83:34:b2:34:7c -> 192.168.20.146 (older)
```

and the historical record ends up in the Device Monitor database.

This also explains the incorrect Offline state, because `is_active` is recalculated using the `last_seen` value of each historical record.

### Proposed fix

I agree that Device Monitor should only pass the newest Hostwatch record for each MAC address to the rest of the scan logic.

One solution would be to do the deduplication directly in SQL, as you suggested.

However, I am currently considering doing the deduplication immediately after reading the Hostwatch result instead. Since the query is already sorted by:

```sql id="lh3gr4"
ORDER BY last_seen DESC
```

the first occurrence of each MAC is the newest one.

For example:

```python id="5ym4vr"
seen_macs = set()

for row in rows:
    mac = (row['ether_address'] or '').lower()

    if not mac or mac in seen_macs:
        continue

    seen_macs.add(mac)

    iface = row['interface_name'] or ''
    vlan = map_interface_to_vlan(iface)

    vendor = row['organization_name'] or 'Unknown'
    if len(vendor) > 40:
        vendor = vendor[:37] + '...'

    devices.append({
        'mac': mac,
        'ip': row['ip_address'] or '',
        'hostname': '',
        'vendor': vendor,
        'vlan': vlan,
        'first_seen': row['first_seen'] or '',
        'last_seen': row['last_seen'] or '',
    })
```

This keeps the Hostwatch SQL simple and also avoids depending on additional Hostwatch database fields for tie-breaking.

The result is that only the first (newest) record for a MAC is passed to Device Monitor, while all older historical records for the same MAC are ignored.

I would still be interested in seeing the exact SQL modification you tested, especially the tie-breaker using the Hostwatch ID. Please post it if you have it available. I can then compare both approaches before committing the fix.

Thanks again for the excellent report and for verifying the behavior before and after the change. Your testing made the cause very clear.

I will include a fix for this in the next Device Monitor update.

For future bug reports and feature requests, it would be even better to open an issue directly on the project's GitHub repository. It makes it much easier for me to track reported problems, discuss proposed changes, attach patches and link fixes or commits to the original report. The OPNsense forum is of course still useful for general discussion and user support, but GitHub Issues is the preferred place for reproducible bugs like this one.

Thanks!
         
Quote from: apg1959 on August 19, 2026, 09:01:35 AMHi,

I'm running OPNsense 26.7.2_2 with Device Monitor v2.1.

I found an issue where multiple historical Hostwatch records for the same MAC address can cause Device Monitor to select an old IP/interface record instead of the current one.

Example

My Windows PC has:

MAC: 7c:83:34:b2:34:7c
Current IP: 192.168.20.111
Interface: igc1 / LAN

Device Monitor initially showed this device incorrectly as:

IP: 192.168.20.146
Interface: IGC1
Offline

I checked the Hostwatch database directly:

sqlite3 /var/db/hostwatch/hosts.db \
"SELECT interface_name, ip_address, ether_address, first_seen, last_seen, organization_name
FROM v_hosts
WHERE ether_address='7c:83:34:b2:34:7c';"

Hostwatch contained three records for the same MAC:

igc1  192.168.20.146  2026-08-15 03:04:46
igc0  192.168.20.146  2026-08-15 11:10:03
igc1  192.168.20.111  2026-08-19 06:06:49

The last record is the current/correct one.

I also confirmed Hostwatch itself was updating correctly. For example, its most recent records included:

7c:83:34:b2:34:7c  192.168.20.111  igc1  2026-08-19 06:08:51

So Hostwatch was not the problem.

Cause

I traced this to:

/usr/local/opnsense/scripts/OPNsense/DeviceMonitor/scan_network.py

The Hostwatch query currently retrieves all matching records:

SELECT
    interface_name,
    ip_address,
    ether_address,
    first_seen,
    last_seen,
    organization_name
FROM v_hosts
WHERE protocol = 'inet'
  AND ether_address NOT IN ('ff:ff:ff:ff:ff:ff', '00:00:00:00:00:00')
  AND ip_address NOT LIKE '169.254.%'
ORDER BY last_seen DESC

There is no deduplication by ether_address.

Device Monitor's own database uses the MAC address as the key:

mac TEXT PRIMARY KEY

Therefore, when multiple Hostwatch records exist for the same MAC, they are processed sequentially and can overwrite one another. In my case, the historical .146 record ended up being stored instead of the current .111 record.

This also affected the Online/Offline status because Device Monitor was evaluating the stale record's last_seen.

Test fix

I tested a local modification to the SQL query which selects only the newest last_seen record for each MAC address, with the Hostwatch id used as a tie-breaker.

After applying the change and running a full scan, Device Monitor immediately changed the affected PC to:

MAC:      7c:83:34:b2:34:7c
IP:        192.168.20.111
Interface: IGC1
Status:    ONLINE

The Device Monitor database then showed:

7c:83:34:b2:34:7c
192.168.20.111
IGC1
2026-08-19 06:18:58
is_active = 1

I subsequently restarted Device Monitor and confirmed the service restarted normally and the Devices page continues to show the correct IP/interface/status.

Suggested fix

I believe the Hostwatch query should deduplicate records by ether_address and select the record with the newest last_seen before passing the results to the Device Monitor database.

This would prevent historical Hostwatch records from overwriting the current record for the same MAC.

The email notification functionality otherwise works correctly for me. I have successfully tested detection of a new MAC address and received the Device Monitor email notification through Postfix.

Thanks — I've attached modified SQL/query if useful.
#3
Good day, I have released version 2.1. which supports Dnsmasq DNS & DHCP.
Again I look forward to your reactions :).

The link is:

https://github.com/hacesoft/opnsense-devicemonitor
#4
Quote from: mooh on April 21, 2026, 10:01:40 PM
Quote from: hacesoft on April 21, 2026, 06:51:56 PMNot every plugin is for everyone — install what fits your needs. 🙂
I didn't mean to belittle your effort. In fact, I appreciate every effort to improve OPNsense.

I just not sure what the size of the target audience for all this host discovery stuff is, yours and the new component of OPNsense. If I want to know what's going on on my network, I ask it directly via SNMP or an all-in-one solution like Unifi or Omada. Only small, unmanaged networks don't already provide that functionality. How many are there using OPNsense?

Hello, I recently switched from pfSense to OpnSense and this option was standard in the system and I use it to check if an uninvited guest is connecting. Or when I connect a device, I find out its IP address. I don't have unifi, or rather I only use AP AC RL and I have a classic L2, L3 switch. So OpnSense will provide me with the necessary information :). I originally wrote the plugin just for myself, but I decided to share it with others... I would like the plugin to be offered as a standard package, but that's too much for me for now :).
#5
Quote from: SteffenDE on April 23, 2026, 07:49:14 AM
Quote from: hacesoft on April 21, 2026, 06:38:33 PMHi Steffen,
The hostname field works as follows: the plugin pulls the hostname from Services → ISC DHCPv4 / DHCPv6 → DHCP Static Mappings — specifically the Hostname field for each entry. If a device doesn't have a static mapping with a hostname defined there, the field will simply remain empty, as the plugin has no other automatic source for this information.
You can also fill in the hostname manually directly in the plugin, but note that this is stored only in the plugin's own database — it does not propagate back to OPNsense DHCP or any other system.
So the short answer: populate the Hostname field in your DHCP static mappings and it will appear automatically.

I have hostnames defined at Dnsmasq because I think ISC is deprecated. So it would be nice to support Dnsmasq too.
Good day, I just converted my home network from ISC to DNSmasq and it will take a while before I get to it and modify the plugin. but don't worry, it will happen :).
#6
Quote from: pc44 on April 23, 2026, 09:42:44 PM
Quote from: pc44 on April 23, 2026, 04:26:58 AMI have an older version of Device Monitor installed and working.  It is great !!!

I am happy with it, but is there a way to update to this new version, or do I need to fully uninstall/reinstall?

Thank you.

Figured it out.  Deleted the existing /tmp/opnsense-devicemonitor folder.  Then downloaded, unzipped, and copied over the new files.  Then just re-ran sh install.sh.

Now up-to-date. ☑️
Good day, exactly as you asked. If you use sh install.sh, everything except the database will be deleted before installation. After installation, the plugin will be started again. If you use sh uninstall.sh, everything will be removed.
#7
Quote from: mooh on April 21, 2026, 03:30:38 PMI would like to see this software rolled into one plugin together with the device discovery service added to OPNsense recently. There's some overlap in functionality.

Then again, unless the information is accumulated over a multi-layer network, i.e. across routers, I could just as well query the network management software for it. I can see how filtering MACs into FW Aliases can be useful if one manages networks on the basis of MAC addresses, but I don't.

The primary motivation for building this plugin was notifications — automatically alerting me (via email or webhook) whenever a new or unknown device appears on the network. That's the core value-add, and it's something OPNsense still doesn't provide natively. Everything else — custom hostnames, clickable URLs, having it all in one place — is convenience on top of that.
An important part of the plugin is also device identification — it works on several levels: hostname (pulled from DHCP static mappings, or filled in manually by the admin), a custom admin note, and vendor identification resolved from the MAC address prefix (OUI lookup). This has been part of the plugin since v1.0.
Regarding merging with the native discovery service: Device Monitor v2.0 already builds directly on the hostwatch database (/var/db/hostwatch/hosts.db), so that overlap has been intentionally addressed. Interestingly, hostwatch didn't exist at all when I started writing the plugin — it was added somewhere between v1.0 and v2.0, and I was happy to take advantage of it. The plugin no longer does its own ARP/tcpdump scanning. A nice case of the platform catching up mid-project. 🙂
On your multi-layer network point: you're right — like hostwatch itself, this plugin only sees devices on directly connected segments. For deeper topologies a dedicated NMS like LibreNMS or Zabbix would be the proper tool. This plugin targets setups where OPNsense is the network edge.
Not every plugin is for everyone — install what fits your needs. 🙂
#8
Quote from: SteffenDE on April 20, 2026, 03:16:58 PMHi,

Nice tool, not perfect yet but provides a good overview.

But what doesn't seem to work at all is the hostname, which is always unfilled?


Steffen

Hi Steffen,
The hostname field works as follows: the plugin pulls the hostname from Services → ISC DHCPv4 / DHCPv6 → DHCP Static Mappings — specifically the Hostname field for each entry. If a device doesn't have a static mapping with a hostname defined there, the field will simply remain empty, as the plugin has no other automatic source for this information.
You can also fill in the hostname manually directly in the plugin, but note that this is stored only in the plugin's own database — it does not propagate back to OPNsense DHCP or any other system.
So the short answer: populate the Hostname field in your DHCP static mappings and it will appear automatically.
#9
Version 2.0 is released today. Completely redesigned :). And it already looks usable :).
#10
Good day, here is a link to the latest version of my Device monitor plugin: https://github.com/hacesoft/opnsense-devicemonitor
#11
Quote from: Seimus on January 02, 2026, 01:27:06 PMWould it be possible to have as well notifications via webhook e.g to support ntfy instances?


Have a nice day, I added support for webhook, ntfy and custom
#12
Quote from: Monviech (Cedrik) on January 02, 2026, 01:29:13 PMHello,

there is a new hostdiscovery service on the OPNsense roadmap that uses a rust written daemon that captures arp and ndp messages via pcap to build a database of known devices.

https://github.com/opnsense/hostwatch

https://github.com/opnsense/core/pull/9354

So something comparable is a core feature soon and integrated into a few components like aliases and captive portal.

So as feedback, you could use the existing sqlite database of the hostwatch service since its in core anyway if you want your own GUI around it.

Have a nice day, it's not yet :), so I'll use my own solution. My plugin can even display devices that don't have an IP address :). And in the DHCP settings, 'Deny unknown clients' is enabled, then I only get the MAC address, which is what I wanted :). And to send the result by email :). If the future add-on works the same or even better, I'll use that, for now I have this :).
#13
Quote from: Seimus on January 02, 2026, 01:27:06 PMLooks interesting, and remembers me on NetalertX.

Few questions here:
QuoteRequirements

    1. OPNsense 24.x or newer
    2. Working SMTP configuration (System → Settings → Notifications)
    3. SSH access enabled (System → Settings → Administration → Secure Shell)
    4. Root password

2. Working SMTP configuration (System → Settings → Notifications)
Would it be possible to have as well notifications via webhook e.g to support ntfy instances?


3. SSH access enabled (System → Settings → Administration → Secure Shell)
4. Root password
Does this work only with a Root account? Or does this work with any active admin account with proper permissions?

Regards,
S.

Good day, it definitely wouldn't be a problem to use a webhook to send data instead of email notifications. I have something similar planned at home, where I will be sending data to a protocol center that I have on my NAS in BSD format (RFC 3164). I have the ROOT account disabled on the firewall, and I have my own Admin account on which the plugin works nicely.
#14

Hello everyone! 👋
I've created my first OPNsense plugin:
https://github.com/hacesoft/opnsense-devicemonitor

And would like to share it with you. It's called Device Monitor - a tool for automatic network device monitoring and detection.
What the plugin does:

🔍 Automatic network device scanning (ARP + DNS)
📊 Online/offline status display
🔔 Email notifications for new device detection
🏷� Manufacturer identification using OUI database
📈 Dashboard with device overview

Technical details:

Python daemon with configurable scan interval
MVC architecture following OPNsense standards
REST API for control
Czech and English translations

The plugin is fully functional, but definitely not perfect. I would love to hear your feedback:

What could I improve?
What features would be useful?
Where did I make mistakes or violate best practices?
Any suggestions for improvements!

I'm open to constructive criticism and looking forward to your insights. Thanks for your time! 🙏
#15
Thank you for your response and recommendations
I'd like to clarify a few things about my configuration:
DNS and hostname:
I'm using internal DNS (firewall.local), I don't have any DynDNS or public domain. I've already created an Unbound DNS override (firewall.local → LAN IP).
Problem resolution:
The problem wasn't NAT Reflection or DNS - the entire issue was a missing firewall rule. The WiFi VLAN was completely isolated and had no access to the firewall, so it couldn't reach the OpenVPN port either. After adding a rule that allows WiFi VLAN → Firewall IP:2000/UDP, everything works.
Security concerns:
Although the problem is solved, I'm not happy with this solution from a security standpoint. By opening a rule from WiFi VLAN to the firewall, I've essentially "punched a hole" in the isolation I originally didn't want.
Original intent:
My goal was to have the WiFi VLAN completely isolated from the local LAN network. Anything from WiFi VLAN that needs access to LAN should only be available through the VPN tunnel. I'm aware that without a valid certificate, nobody can connect to OpenVPN, but I still don't like that WiFi devices have direct access to the firewall IP.