Hostwatch - high disk writes

[franco]

These typical UFS issues mainly are from unclean shutdowns regardless of where the corruption occurs.

If you have a process that is writing while the power goes off there will be an error for it. And if the process is writing all the time the chances are pretty high it's going to catch that one.


Cheers,
Franco

[pfry]

We're recording the last MAC address for any IPv4 and IPv6 we see. If the MAC changes that's considered a "movement". In some environments this happens very rapidly and thus the service constantly registers the changes.

OK, thanks. This is just for virtual interface setup/teardown? Interesting. The behavior sounds... less than ideal. Not something I (and apparently others) would expect.
(The overall issue reminds me to always control information rate, unless a spew is intended.)

[GreenMatter]

These typical UFS issues mainly are from unclean shutdowns regardless of where the corruption occurs.

If you have a process that is writing while the power goes off there will be an error for it. And if the process is writing all the time the chances are pretty high it's going to catch that one.

It is freshly installed system (UFS, because of VM's disk is on ZFS) with restored config and updated to 25.7.11. It didn't experience any unclean shutdowns... BTW, I couldn't finalized installation (was stuck at "preparing target system") if I imported config during installation. I had to install Opnsense, configure lan interface and restore config using webgui...

[nicholaswkc]

How to disable loggin of hostwatch or can it be disable?

[Patrick M. Hausen]

Interfaces: Neighbors: Automatic Discovery

[yeraycito]

In my opinion, the plugin is useless; it should work like this:

https://github.com/netalertx/NetAlertX

[Monviech (Cedrik)]

Your opinion does not see the full picture, the hostwatch daemon was mostly designed to discover IPv6 addresses, so that for example an IPv6 only captive portal is possible in the future.

Any consumer that needs a full picture of IPv6 addresses now has a fast sqlite database to work with.

The other things that happen with the data can still be fleshed out over time.

[nicholaswkc]

Interfaces: Neighbors: Automatic Discovery

Thanks.

[xpendable]

I also had higher memory consumption with an increasing swap file size as well. Noticed that hostwatch was enable on all interfaces, as such it was logging IPs on the WAN interface as well. From my understanding this is targeted at keeping track of IPs on your local network, so shouldn't the WAN interface be excluded by default?

Odd thing is I disabled the service, selected all interfaces except WAN and re-enabled the service. However now the service refuses to start and nothing in the hostwatch logs says why. The system log reports:

Code Select Expand

[meta sequenceId="2"] <6>[132593] pid 22972 (hostwatch), jid 0, uid 0: exited on signal 6 (no core dump - bad address)
So for now I just left it disabled since it seems to provide relatively little value.

[Maurice]

@xpendable

https://github.com/opnsense/hostwatch/issues/13

[OPNenthu]

[...] the hostwatch daemon was mostly designed to discover IPv6 addresses, so that for example an IPv6 only captive portal is possible in the future.

Any consumer that needs a full picture of IPv6 addresses now has a fast sqlite database to work with.

The other things that happen with the data can still be fleshed out over time.

Ok, it just clicked.

Can this also track IPv6 temporary addresses?  If it will become possible to create a host alias that updates dynamically from ND data, or even update DNS so that firewall/unbound logs resolve to a host using privacy addresses...

Tempering my excitement :)

[Monviech (Cedrik)]

Yes it can track temporary addresses because of this:

https://github.com/opnsense/hostwatch/issues/2

[OPNenthu]

Nice! 👍

I don't follow GitHub that closely and totally missed this.

[GreenMatter]

Updated to 25.7.11_2.
Unfortunately there's only slight difference in i/o demand created by hostwatch.
top -S -m io -o total
last pid:  8428;  load averages:  0.39,  0.41,  0.37                                                                                                                           up 2+17:07:59  11:52:30
144 processes: 2 running, 140 sleeping, 2 waiting
CPU:  5.1% user,  0.0% nice,  2.9% system,  0.5% interrupt, 91.5% idle
Mem: 343M Active, 6483M Inact, 1559M Wired, 670M Buf, 3881M Free
Swap: 8192M Total, 8192M Free
  PID USERNAME     VCSW  IVCSW   READ  WRITE  FAULT  TOTAL PERCENT COMMAND
92104 hostd       2707     37      0   2640      0   2640  99.21% hostwatch
 7034 root          53     54      0     19      0     19   0.71% python3.11
   16 root         548      1      0      2      0      2   0.08% bufdaemon
    1 root           0      0      0      0      0      0   0.00% init
97153 unbound       47      1      0      0      0      0   0.00% unbound
    2 root          56      0      0      0      0      0   0.00% clock
 5314 root          12      6      0      0      0      0   0.00% ng_queue
 1474 squid          0      0      0      0      0      0   0.00% security_file_certg
    3 root           0      0      0      0      0      0   0.00% crypto
10115 root           0      0      0      0      0      0   0.00% ge


iostat -x 2
                        extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
da0            0     105      7.7   4048.1     0     1     0     1    0   0 
da1            0       0      0.0      0.0     0     0     0     0    0   0 



Plus I wasn't able to start hostwatch when I handpicked interfaces, works only when "All" is selected. When trying to start it in cli:

service hostwatch restart

hostwatch not running? (check /var/run/hostwatch/hostwatch.pid).
Starting hostwatch.
thread 'main' (116664) panicked at src/main.rs:53:79:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Abort trap
/usr/local/etc/rc.d/hostwatch: WARNING: failed to start hostwatch

[franco]

It seems all issues have now been recorded in https://github.com/opnsense/hostwatch/issues and don't need to be reposted.

Let's give it a bit of slack to be resolved.


Cheers,
Franco