Messages

Yes, that's perfectly normal. And also yes, the consumer routers don't even bother to create such logs, as they're not useful to their intended audience (since the average user couldn't do anything about it, anyway) and in fact more likely to cause the effect you experienced. ;)

Skimming these screenshots, there seems to be only a single instance of an actual "port scan" in the traditional sense, which is the one coming from 185.246.128.192. All others either are "trickle scans" (which is unlikely, you only use these when you have a specific mark and know they're looking out for scans) or just misdirected connection attempts by legitimate users. These come from outdated dynamic address info, which you get literally on a daily basis with IPv4 (the entire point of using DynDNS services). Most are single connects, which could be looking for one specific service to (ab)use (probably some IoT stuff, which has a high probability of being both outdated and unprotected. Maybe a botnet trying to find peers.). But then again, the ports wouldn't differ so much (IMO). If you're curious, you could try looking up these ports to see what they might be after. Or it might just be what happens when something like bittorrent is started with a cache more than a few weeks old. And then look at the guy on 45.142.193.191, several attempts to the same port (52073). Probably someone trying to connect to their buddies self-hosted game server. Or someone about to find out that their own DynDNS had failed, as happened to me more than once. :) You'd probably see a few pings afterwards the first time they experience that. ;)

So, as has been said: make certain to only have ports open that you know about, and then make certain these services are protected as good as you can manage, and also run only when they actually need to run. The really dangerous stuff would appear in green, indicating something was allowed through. So you'd filter for unexpected connection attempts to the services you do have open and check the logs of these services. You could also try to find any suspicious activity from their pattern (if there is any), and then ponder what could possibly be done to prevent that.
For a while I was wondering about the lucky chance of someone probing a port that has been opened for replies to an outgoing request, but that's why firewalls track those (this is the "state violation" part of the message). Note that if you open some port for games or such you'll not be in immediate danger when the game isn't running: the packet will be allowed into the LAN, reach your PC and then go poof. However, your system will send an ICMP error message, so the attacker knows that a system is reachable on that port, but has nothing there. So it could be used to map you out and plan an attack based on that info, but attackers are lazy and will just move on to find easier prey. Especially with your IP address changing daily, they'd lose track of you unless they are tracking your IP. In that case, you should indeed be worried, but of course you'll never know about that until it's too late (or not even then).

So, the net sure has become busier than a decade ago, but that's a given with the number of devices and people connected. More noise, and more potential victims.

Ransomware usually arrives through email. Basically, anything with an attachment definitely is, and anything without has links to either phishing or more malware. This has always been the most successful attack, and its likelihood of working against you increases with every account you create: more places to steal information from, more information to be stolen, more context to create a believable scam. Currently, there seems to be quite the success with "Payment declined: your cloud data is in immediate risk of being deleted". Of course this works better than the "African prince inheritance fund" scam, because so many people made the mistake of using cloud storage services. Yes, I think it's a liability for businesses, too.

BTW: please correct me if any of my conclusions or reasonings are flawed!

This is very similar to how it is done on contemporary smart TVs, and the ability to move multiple rules does has its advantages over moving single lines only. It's just that the visual representation is harder to use, mostly because you need to read from anywhere in the row but can only access the button in one small place. TBF, without pulling a lot of stunts with JavaScript and friends, I don't see a better way to do it in HTML, and I'm not fond of JS and friends.

To add: without knowing the Cisco device, I suspect that it used to be the "all in one" internet access device every ISP hands out. As coffeecup25 hinted at: OPNsense is not a full replacement for that device, because it lacks the actual switch. What your multiport OPNSense box is equates to a PC with multiple NICs. What your Cisco thing is equates to a PC with 2 NICs and a switch connected to one of these NICs. The first is much more powerful in terms of potential functionality than a simple switch, but that means it's not (at all) optimized for use as a switch. (BTW: yes, it is perfectly conceivable to have a switch integrated on a NIC, which you could place in a normal PC alongside its built-in NIC to make it work just like the Cisco thing).

So now you would need an external switch. However, you can relegate your Cisco to a switch. To make it so, you would need to:

1) create in OPNSense a configuration that mimics what is present in the Cisco, including the IP addresses(!), DHCP rules, etc.
2) disable all services on the Cisco, especially DHCP, then change the IP/subnet of the Cisco to something else (this way it will still be accessible on your LAN if your wish). You could also put it into another subnet so that it will only be accessible if you change your LAN IP appropriately for configuration purposes. Since it will not receive any updates anymore this might add a little obstacle to the casual attacker poking around your LAN. The downside of leaving any "Smart" device, especially unmaintained ones, on the LAN is that it increases your attack surface, plus the power draw for the now unused CPU and WAN parts. But you'd get these same issues with a smart switch of the same age.
3) if currently done any different, plug the OPNsense box into one of the LAN ports of the Cisco thing (NOT the WAN port)

Optional: put a sticker on the Cisco noting it's IP, because the next time you'll consider logging into it will be in 4 years. ;)

Fällt die Verbindung zum RZ auf beiden Standorten aus? In dem Fall wäre es ja dasselbe Verhalten bei zwei Geräten, was eigentlich nicht auf Hardwarefehler hindeuten würde. (Ich gehe mal davon aus, dass vor dem Konfigurationsexport die Softwareversionen identisch mit den Zielboxen waren.)
Vorstellen könnte ich mir, dass die originalen Konfigurationen, die ja sicherlich mehrere Releases und damit ggfs. Formatänderungen hinter sich haben, beim Export nicht vollständig ausgegeben werden. Das könnte man testen, indem man die exportierte Konfiguration auf der alten HW zurückspielt, dann sollten zumindest nach einem vorherigen Reset auch dort dieselben Probleme auftreten. In dem Fall müßte man dann wohl die Konfiguration identisch neu erstellen. Sowas sollte natürlich nicht vorkommen, aber...
Obwohl unwahrscheinlich: die neue HW hat ja auch neue MAC-Adressen. Wenn da irgendwo alte Filter, feste Routingregeln oder IP-Adressvergaben drauf basieren (DHCP), gäbe das sicher Probleme.
Fehlt evtl. ein Plugin für irgendwas? Bei einem Konfigurationsimport werden die ja nicht automatisch installiert, sondern in der Liste als "fehlt" oder so angezeigt, was nicht direkt auffallen muss.

Auch die LL und die Global Adresse bekomme ich jeweils bei den Clients.

Schau mal in den Firewall-Logs, ob die Pakete nicht evtl. geblockt werden. Die Kombination von 1 und 4 bedeutet ja, dass nur die OPNsense nicht raus darf, alle anderen im Zwischennetz aber schon. Entweder blockiert sich die OPNsense also selber, oder sie hat eine andere IPv6 / Subnetz / VLAN als alles andere im Zwischennetz. Oder sie wird von der FB blockiert.

Das müßte mit Monit gehen. Also zuerst Monit einrichten mit Email usw., und dann einen Test erstellen:
Erst in "Service Test settings" einen neuen Eintrag:
Name -> File content changed
Condition -> Checksum failed
Action -> Alert

Speicherm

und dann in "Service Settings"
Name -> DynDNS_change
Type -> File
Path -> /var/cache/ddclient/ddclient.cache (!Das kann auch woanders sein, mußt Du mal mit "find ddclient.cache" suchen!)
Start ->
Stop ->
Tests -> File content changed (oder wie auch immer Du die oben erstellte Bedingung genannt hast)
Depends -> nothing selected
Description -> Check for change in ddclient IP cache

Soweit jedenfalls die Theorie, aus dem was ich auf die Schnelle an Docs gefunden habe. Andere Backends werden ihre Files anders nennen, aber das Prinzip bleibt dasselbe. Das "native" Backend aus OPNS macht das sicher auch so, aber das Script müßte ich aber auch erst suchen und dann den Dateinamen extrahieren. Evtl. kennt den hier ja jemand schon und erspart uns die Sucherei?
Wie man da jetzt noch die IP in die Mail bekommt, fällt mir spontan allerdings nicht ein.

Vielleicht wäre ein externer PEO Injektor die bessere Wahl für Dich? Klar ist es ein Gerät zusätzlich, aber damit bist Du bei der Wahl des Switches, inklusiver zukünftiger Änderungen, wesentlich freier.

Was das schwache Powerlan angeht, könntest Du die Adapter nochmal resetten und am tatsächlichen Anbringungsort neu verpaaren. Oft werden die nebeneinander gesteckt, gepaart, und dann verteilt. So oder so kann sich an der Verbindung untereinander was ändern (und sei es nur ein unschuldig aussehendes neue Steckernetzteil), was die ganze Signalaushandlung zunichte macht. Da kann dann eine Neuaushandlung Wunder wirken (so gesehen: von 14MBits/s zu über 200MBit/s). Die Verbindungsqualität kann man mit den Powerlan-Managerprogrammen angucken, leider aber keine Neuashandlung anstoßen oder sonstiges Management betreiben.

Generell gilt: das Einzige, was ein festverlegtes LAN-Kabel übertrifft, sind zwei festverlegte LAN-Kabel. Wenn Du also diese Chance hast: nutze sie, und verlege so viele wie möglich, wenn Du einmal dabei bist.

Denn egal was man tut: die schlechte Bandbreite kommt nicht von dem einen Port an der OPNsense Box, sondern vom Powerlan und / oder dem WLAN (hast Du ja auch so beobachtet). Eine auf 1Gigabit ausgehandelte WLAN-Verbindung bricht bei ernsthafter Nutzung aber auch schnell ein, und je mehr Parteien mitfunken (eigene Geräte oder die der Nachbarn), umso schneller wird es schlechter. Das ist bei Powerline nicht anders, nur, dass da jede Lampe, jedes Steckernetzteil, Fernseher, Computer, etc. ggfs. für Störungen sorgen. Daher ist es auf jeden Fall sinnvoll, möglichst viele Geräte möglichst direkt und per Kabel anzubinden. Da würde aber kein Unterschied zwischen dem einzelnen Port an der OPNSense Box und der Bündelung herauskommen, weil entweder der Verkehr gar nicht über die OPNSense Box läuft (z.B. für ein NAS), oder durch die Internetanbindung begrenzt ist, und bei Beidem zusammen ja eigentlich auch nicht (da begrenzt dann eher die Verbindung am Rechner). Die Portbündelung würde IMO nur dann etwas bringen, wenn 1) Deine Internetverbindung deutlich schneller ist als der Port an der OPNSense Box und 2) Deine Geräte entweder gleichzeitig oder einzeln diese Bandbreite auch absorbieren können. Deiner Beschreibung nach ist ein Gerät (der Desktop-PC) Hauptnutzer der Bandbreite und die anderen sind Nebenschauplätze. Es wäre wahrscheilich am besten, das Hauptgerät direkt anzubinden und so dem Rest das ganze W/PLAN zu überlassen, die VLANs sind dafür eigentlich egal.

Der Hauptgrund, trotzdem einen Managed Switch zu nehmen wäre die Tatsache, dass an jedem (derzeit noch freien) Port eher früher als später ein Gerät hängen wird. Und wenn der dann keine VLANs kann, ist das natürlich schlecht.

Also, some devices look for firmware updates, can order consumables in advance a.s.o., let alone collect statistics and push that to the manufacturer without anyone knowing.

There are good reasons to put "smart" devices into a separate VLAN.

Indeed, that's what I was implicitly hinting at, and rather than ordering consumables I'd rather have it send a mail to the admin instead, and use a local NTP server, but I accept that the default configuration might be catering to the person who plops down the device, pops in some cables and then expects it to work until it's decommissioned. Of course, this affords privacy and security only to those select "elites" who at least know what might be happening and therefore might try to prevent them.

Does anyone know where scope checking is done WRT this diagram? I'm referring to the part that'll generate a "Destination unreachable: Beyond scope of source address" message when a link-local tries to route to global space? It seems to be done before pf, because it also applies if I set the firewall to disabled (check "Disable Firewall Disable all packet filtering."), which (I think?) would bypass pf entirely (is that assumption correct?). Reason: I'd like to selectively change this behavior using pf rules, so that the exceptions exist only as long as the firewall is enabled, and only for the protocol(s) specified.

I could find precious little documentation on the entire subject of address scoping; almost all results are about the scope ID, which is basicly the %ifX suffix, not the "address scope" I'm referring to, and next to nothing about the handling and manipulation of the link- and site-local scopes themselves. There's more for Linux, but none of that can be applied to BSD.

Addendum: after experimenting some more with it, it seems like the redirection step actually comes after the inbound filtering, which is consistent with RFC4890, so that one can manually block this sort of traffic for logging purposes. However, NAT only happens in the out section, and between that and the in section the automatic denial already happens, so it is not possible to match on the traffic in order to NAT it (so between 3 and 6, possibly between 3 and 4 even). So, without manually reconfiguring the kernel (which obviously is not acceptable both in terms of maintainability and opportunities for errors), I am denied this fail-safe mechanism, whether it would actually work as intended or not. I feel there lies a missed opportunity for fallback security in this. The only way around this would be to set up a transparent filtering bridge, which however would rely even more on proper filtering and doesn't afford any protection at all, just like disabling scope checking altogether.

Would be interesting to know if the printer would also work normally if it had no gateway set at all. IMO a printer has no business outside its local network, anyway, so doesn't need / shouldn't have a gateway.

At random intervals my network seems to go down. I cannot access anything on LAN or WAN when this happens. but there is nothing in any logs to explain why this happens.

Access what from where? From what you write it seems like a client or switch issue ("cannot access anything on LAN or WAN"->not even local servers / printers)?

OPNsense does nothing on its LAN side except DHCP, DNS and such services, but cannot by itself block you from accessing other devices on your LAN, except if you use the OPNsense box as "switch" by bridging...

I've just finished reading that thread, and ZFS writing metatata constantly indeed was new to me. However, (I edited it in but it overlapped), with the -nano images you're always on UFS, which I find a reasonable choice given the type and size of these drives, and especially if ZFS won't even help in that department, anyway.
Right, so as option 2) is out unless I delay updates to once per year which obviously is a bad idea, I'd need to see if option 1) can somehow be made to work, still -nano and thus UFS.

In the linked thread there were lovely graphs that were made with "CheckMK", but sadly markus.tobatz didn't say how to install that on OPNsense, but in the end they're not much different from what iostat does.

Thanks for the consideration! I get about 14MB/s writing to the thumbdrive with dd, whereas the update process reports around 0.1MB/s. I tried the dd with bs=64k, but thanks to your hint I looked it up and the chunk size can be as low as 8, 16, 32, or 64 bytes for "full speed" drives, and up to 512 with "high-speed" devices. Even if read-modify-write cannot be avoided, it would perhaps suffice if the writes were bunched, so small chunks might still be OK if it were aggregated. Of course, this only works if the write cache also serves as read cache so that pending changes would not have to be flushed before a read can commence. IDK if this would be normal behavior if such a cache were to actually exist.
I only found mention of the "logging/nologging" parameter for ufs mounts, but that looks like it'll just add more writes by first writing what it plans to write, and then actually writing that, and then erasing that from the log. Sounds similar to the journalling of ext4 or the ZIL in ZFS, but with the latter I could possibly trick it into using a dedicated ZIL drive that itself is a RAM disk. Obviously this would be a really bad idea for a file server, but as a drive cache might be acceptable, but still wouldn't mean the writes actually do get bunched. Plus, using ZFS on a puny 4/8GB USB thumb also seems like a bad idea, and if I went for a proper internal SSD, I might just as well forego all these considerations because that does caching and has tons of write cycles plus heaps of spare sectors, TRIM, etc., anyway. So currently I'm using UFS, but there isn't even a choice anyway with -nano. :)

[nano]

I've set up OPNsense-nano in two ways and both have issues regarding drive wear:

1) using an external USB enclosure.
- the issue with this is that while disk writes don't matter, this being the root fs doesn't allow the drive to spin down, even on an idle system. I'd have expected that, it being nano, all access is to the ramdrives, while OS data / programs will essentially be cached. The system has barely any configuration and no traffic whatsoever, but still the drive never powers down, even after a day of just sitting there. When the drive is merely attached to the system, but not mounted, it does power dosn after 30 minutes or so, which would be fine.
Problem: this being a mechanical drive (laptop drive), actual spinning time will likely increase wear and increase power usage, both of which I wish to avoid by it spinning down once booted.

My preferred resolution would be to just cache everything required during normal operation and not touch the drive anymore, allowing it to sleep until something unusual happens.

2) using a USB attached thumbdrive.
- this configuration is efficient until there is an upgrade. The way the upgrades are done, the drive keeps being hit with small bits of writes, which, besides dragging performance down to abysmal levels, forces the drive to repeatedly erase a section of flash, write a single byte, erase it again, write the next byte, for hours on end (literally). Combined with the nonexistent wear levelling and trim support of these things, I expect this to wear down the device in no time. The drives have no internal cache whatsoever that could be enabled by camcontrol or smartctl (they don't even speak SMART).

My preferred solution would be to have a write cache of, say, 100MB, to and from which all sequential modifications are done, and only flush that to the drive about once per minute, give or take.

I haven't found any way to do option 2 on BSD (maybe with ZFS), and for 1) I don't know enough about both BSD and OPNsense to evaluate at all. Thus far, the only option would be to use a recent SATA SSD, but the external / USB solution is so elegant and flexible in all other respects that it would be a pity to ditch it, plus the SSD would be horrendous overkill.

The image is stored in every config backup file, so you'd not want it to be large filesize-wise. The widget will assume any height, but the width is fixed by the column you put it in, so ideally the image would be an exact match width-wise, but since there is no real way of telling (the widget has a border and a frame, for example). I'd go with something that is slightly larger than what the widget will be, but depending on the source it may look bad scaled. The file size strongly depends on the combination of format and image data, so ideally you'd pick the format that packs the image in the smallest file, like png for line drawings or jpg for pictures with many hues. I'd assume the rendering and scaling is done by the browser in the end, so I'd not use anything unusual, but webp format therefore is OK unless you use old browsers.