How do I know if a non-official plugin is legit, and won't do somehting bad?

Started by Retired Miner, November 06, 2026, 07:34:59 AM

Previous topic - Next topic
I am not and was never an IT professional, a hobbyist in retirement is a good description.

I saw posts (on reddit) about an interesting plugin named os-loupe. What it claims to do is very interesting: Loupe records which device talked to which site name and how much, what was the Device X doing at 2 a.m.? 

It is not listed on the plugins list. You have to run two fetch commands from the console to make it appear as a plugin, the fetches come from github. 

This is where I am 100% lost, if something is not an official plugin how do you know if it can be trusted?

The GitHub webpages look nice and polished (in fact, a little too slick and too polished). They mention AI in quite a few places, which to me is a red flag.  Realistically I don't know what I am looking at in GitHub, or what to look for, under all the source code, docs, workflows, tests and tools.  There could be a few lines of code buried deep in there that send everything to the dark web and I wouldn't know it.


> This is where I am 100% lost, if something is not an official plugin how do you know if it can be trusted?

I think this is the core problem: we cannot by default.  That's why we started the community plugins so that we can have review, discussion and bugfix capabilities.  But this was over a decade ago.  Nowadays with AI assisted large blobs of code we can no longer screen submissions for official plugin inclusion.  People just tend to publish it privately anyway because it beats going through a review process (it works, right?!). Whether that's with the source code open or not doesn't really matter; for one you can read the source code in the install archive and secondly there's still a lot of code to sift through which is just unlikely to be looked at by a human.

The other issue we see is that plugins were mainly meant to accommodate a (known) third party software for integration.  Now the world is being reimplemented in Python scripts in a heartbeat so you don't even have that trust boundary anymore.  Naturally, we don't really like to see these in the plugins because it puts the full maintenance load on the project once it has been accepted, which it can't be because it's not battle proven in the first place.

And, yes, it looks very pretty: code style, documentation, websites, communication.  It is slowly becoming a red flag in this day and age.

FWIW, that's only my opinion.  There's good things in this, but it still takes (human) effort to curate if this is something to be shared with the community.  Otherwise everyone will just have to write their own plugins for themselves with the help of AI.  This would be a bleak future.  ;)


Cheers,
Franco

Quote from: franco on November 06, 2026, 08:29:26 AMwe can no longer screen submissions for official plugin inclusion
Sounds like the review process is not available anyway?

> Sounds like the review process is not available anyway?

Why not check the facts at https://github.com/opnsense/plugins/pulse?period=monthly

QuoteOctober 6, 2026 – November 6, 2026

52 Active pull requests
96 Active issues
32 Merged pull requests
20 Open pull requests
66 Closed issues
30 New issues

Excluding merges, 30 authors have pushed 75 commits to master and 101 commits to all branches.

On master, 148 files have changed and there have been 3,776 additions and 962 deletions


Cheers,
Franco

Quote from: Retired Miner on November 06, 2026, 07:34:59 AMLoupe records which device talked to which site name and how much, what was the Device X doing at 2 a.m.?
So basically a DNS Query Log Overview ?!

My Pi-Hole does that too...

No need to polute OPNsense with all kinds of AI coded crap that does not belong on it! :)
Weird guy who likes everything Linux and *BSD on PC/Laptop/Tablet/Mobile and funny little ARM based boards :)

Quote from: franco on November 06, 2026, 01:53:50 PMWhy not check the facts at https://github.com/opnsense/plugins/pulse?period=monthly
I think you know that was not what I was asking - not least because I am one of the contributors to those issues and PRs. I don't think the passive-aggressiveness is warranted.

Your statement in the earlier post suggested the dev team would no longer accept new plugin submissions because you do not have capacity to review them. Is that the case or not?

The answer is that you don't know, but you are not alone. The guy who created os-loupe hasn't read the 28 thousand lines of the plugin, nobody has...

This is the new normal.

That being said, a good first step is to drop the whole repo to an LLM and ask about it.
potatonetworks.com

We actually reviewed new plugins and merged them like the netreflector and cloudflared plugin. Also suricata ruleset (julioraup antiphishing).

We also respond to new plugin proposals and give them a look, but most of them are either very niche or they are just lots of python code.

Also for most possible contributors the fun is in quickly creating something new, not really in maintaining it for several years.

When it was still harder to create something one could assume the creator had some passion for what they did at least for a while, cause it took some serious effort.

But now I can just ask AI myself to look at Y and create X from that for me personally. I can completely cut out any middle man, the AI can build personal software for exactly my needs. Why even dump it into a repo if it is like a tailored suit for yourself?
Hardware:
DEC740

Thanks Cedrik, appreciate the constructive response.

Why dump it into a repo or even propose for it to be included in the official plugins? Same reason as would always have applied - to contribute to the OPNsense community and (in the latter case) give people the confidence to use it.

Yes, AI lowers the bar for people to create contributions, whatever the quality may be (and it is not all bad). But AI is here to stay. I know the OPNsense dev team even uses it for contributions.

Yep, we also use AI. Like every tool it depends on who uses it and how.

For the plugin repo we want thin wrappers around existing freebsd ports, some applications are also in opnsense ports directly (like my ndp proxy, the new netreflector, the core software hostwatch etc...)

There's a bit of a line to tread sometimes, but imo lots of python scripts, or essentially any kind of business logic as scripts inside the plugin turning it into the application itself makes it very poor to maintain in the future.

Whats also not needed are services that pretty much collide with that opnsense is a firewall, I saw a thread of somebody wanting to provide an iscsi fileserver on it just a few days ago. Well, of course you can, but its unreasonable.
Hardware:
DEC740

Adding to that, whats also bad to maintain is software that uses lots of different providers.

I went down that road with dns providers in caddy but there are so many and they all have some problem at some point I just dumped it like a hot potato and only cloudflare is left. Acme Client also shows just how much is dumped into it. DNS providers are such a mess.

Thats also a problem domain for anything really that uses provider specific accounts, like messaging, arcane vpn protocols, etc. You're just going to get mad maintaining something where user X wants provider Y added and then provider Y stops working like 3 months later cause they change their API again.

Personal opinion though, cause I was burned myself by that maintenance load before.
Hardware:
DEC740


Quote from: nero355 on November 06, 2026, 03:26:22 PM
Quote from: Retired Miner on November 06, 2026, 07:34:59 AMLoupe records which device talked to which site name and how much, what was the Device X doing at 2 a.m.?
So basically a DNS Query Log Overview ?!

My Pi-Hole does that too...

No need to polute OPNsense with all kinds of AI coded crap that does not belong on it! :)
Sorry, I'm lost on this.

If you are asking if it is "a DNS Query Log Overview?" then I can't tell, I have no useful insights, or even an educated guess, as to a yes or no answer.

If "My Pi-Hole" does that, provides the same functionality, then maybe I start reading up on it.

Quote from: sopex on November 06, 2026, 08:28:33 PMThe answer is that you don't know, but you are not alone. The guy who created os-loupe hasn't read the 28 thousand lines of the plugin, nobody has...

This is the new normal.

That being said, a good first step is to drop the whole repo to an LLM and ask about it.

Thanks, I did this. Used google a i. I had to ask (push back) several times to get it to parse through all the code as opposed to regurgitating and rewording what it read in the text documentation.

It came back with...

What Found Analysing the Code
.
.
.
Conclusion

The code uses standard Python socket logic exclusively to talk to internal loopbacks, Unix sockets, and local Home Assistant servers. There are absolutely zero outbound tracking vectors, cloud telemetry loops, or hardcoded external endpoints hiding within the .py source.

It listed Verdict = Safe for these three categories Home Assistant Integration (hass.py), Firewall State Tracking (pfstate.py), BPF Packet Capture (packet.py).

Even with that I'm still not sure I trust it. But based on the above comment 'reimplemented in Python scripts in a heartbeat so you don't even have that trust boundary' I plan to stay away until I can read up some on all this.