Messages

flowd_aggregate regularly dies. I had to create a monit task for it to restart and keep it alive.

Update, for the record, in case someone else has similar problems: I flipped the 2 SSDs between my APU2C2 and APU2C4, now the latter is the router running OPNsense. It's been stable since then.

Perhaps it's an issue with the USB subsystem of the boards, as since the swap my ZTE MFE831 USB LTE modem crashed several times. It is basically a screenless Linux/Android data-only mobile phone which can share its cellular connection via USB.

Interestingly, the previously used APU2C2 has been rock solid since then with Alpine Linux. I was running Linux kernel compilation for several days using 5-6 threads to load all cores and the memory subsystem, but it could took the load without breaking any sweat.

Well, even after upgrading to 20.1.6 it is still an issue.

Basically nothing gets logged remotely until syslog-ng starts up. I understand it from a technical point of view, but it's still not fortunate for a device/OS which has to do anything with security.

Are there any plans for 20.7 (or later release) to rework the logging subsystem?

And... I do not have syslogd on any potential receiving server, but only systemd-journald. Can I use that also?
Thank you.

http://man7.org/linux/man-pages/man8/systemd-journald.service.8.html

It cannot receive remote logs, but can forward its own journal to syslog.

Just install syslog-ng on a *nix OS of your choice and forward everything to it on port 514. Here's a quick example snippet you can use on the receiving side:

source s_network {
    network(port(514));
    network(port(514) transport("udp"));
};

template t_remote {
       template("${ISODATE} ${HOST} ${FACILITY}.${LEVEL} ${MSGHDR}${MSG}\n");
};

destination d_networkhost { file("/var/log/host-${SOURCEIP}_${YEAR}${MONTH}.log" template(t_remote)); };

log { source(s_network); destination(d_networkhost); };


It'll create a listener on both UDP/TCP port 514 and store the incoming logs at /var/log/host-A.B.C.D_YYYYMM.log for each host. I use ZFS with compression on my loghost, so I do not rotate+compress any logs from a cron job.

Just my 2 cents.

OPNsense's PPPoE performance is notoriously bad when you have a weak CPU. Probably this could explain your woes. What hardware do you run it on?

This is alas a FreeBSD legacy and also affects Pfsense. Even if you have capable NICs in the router, when the CPU is underpowered, PPPoE will suffer. Look up PC Engines APU2 related topics here, Pfsense issue 4821 or FreeBSD bug 203856 for examples.

I'd try a Linux-based distrib on the same hardware, like a current OpenWRT to see if the problem persists.

Hello,

I've been having stability problems with my APU2C2-based router (it crashes and reboots randomly) and after one of these occasions the Reporting: Traffic screen stopped working properly. The moving graph is fine at the top, but no matter what I select at the bottom lines (Interface, Sort by, Filter, Display, Top) nothing is displayed below the "Host IP   Bandwidth In   Bandwidth Out   Total In" line. I tried to clear RRD Netflow and Insight data, but it did not help - though I'm not sure where it gets the information from.

Is there any known trick I can do to reset it to a default state? It's not a problem if I lose historical data.

Hello all,

I'm still chasing why my APU2C2 resets randomly (see https://forum.opnsense.org/index.php?topic=16351.0), hence at some point I set up remote logging. Very simple: one rule which forwards EVERYTHING to the loghost.

The problem is: NOTHING gets logged until syslog-ng starts, e.g. boot dmesg.

Here's a log snippet, today morning I restarted my router manually (tune something in the BIOS):

2020-04-11T10:14:29+00:00 auth.info sshd[31201]: Received signal 15; terminating.
2020-04-11T10:18:45+00:00 syslog.notice syslog-ng[91955]: syslog-ng starting up; version='3.25.1'

I don't think that's the right way to do remote logging.

Did I set up something incorrectly or is this feature simply missing form OPNsense?

Hello all,

I have an annoying problem with my router: it randomly crashes from time to time. Sometimes it can go for almost a month without problems, sometimes it reboots after a couple of days. All 20.1.x versions showed this behaviour.

There are no infos at all in the local log files. I set up remote logging to another APU2 running Linux but to no avail, as there were no usable infos, either. Interestingly, the bootup kernel messages were not logged on the loghost, either, but I might need to tune something for that.

I don't think it's caused by overheating, as the CPU temp graphs show constant temperatures between 60-62°C. Of course it could still be a another hardware problem. There's a 16GB mSATA card installed, the internet uplink is provided by a USB LTE stick in PPP mode and all the 3 igbX interfaces are in use. The BIOS is fairly receny, 4.11.0.2.

Another somewhat disturbing thing is that these crashes are not visible in lastlog. Perhaps it's a BSD thingie (I'm mostly used to Linux and Solaris form my jobs and am fairly new to BSDs). The last crash happened today morning at around 8:10 CET - between the 2 topmost lines in last's output.

root       pts/0    A.B.C.D           Sun Mar 22 10:20   still logged in
root       pts/0    A.B.C.D           Sat Mar 21 22:22 - 22:24  (00:01)
root       pts/0    A.B.C.D           Sat Mar 21 06:45 - 06:51  (00:06)

I know it's pretty much trying to catch a black cat in a dark room at this point, but perhaps others also experienced this. Does anyone perhaps have any ideas where I can start to look for more clues?

I also thought about setting up logging for the serial console, as I have a FreeNAS box close to the router which I use for console access with a USB-serial converter.

Unfortunately I cannot trigger this behaviour and I couldn't correlate it to other events, either. This morning it happened while my family was still sleeping :) , so there was basically no traffic and load on the router.

Any help is much appreciated.

Gabor

do-ip6 is relatively dangerous, we tried to play with it for a bit:

https://github.com/opnsense/core/commit/53024b5c159c

Only use it if you do not have IPv6 in your network at all. In which case you can disable IPv6 from Firewall: Settings: Advanced uncheck "Allow IPv6" and do-ip6: no is set automatically.

I don't want to disable IP6 completely, as I have ULA addresses on my home network (in the past I played with a HE tunnel, but the closest gw is in either Prague or Budapest and adds too much delay to the RTT).

I think I had a similar one solved via System : Settings : General : Prefer IPv4

It has been set since the beginnging, but alas did not help.

Thanks mimugmail for the suggestions, I've solved it meanwhile. It's a mix of PEBKAC :) and probably bad defaults.

Unbound tries to connect via IP6 even when it's not available on WAN. My ISP is Drei in Austria and I use a USB LTE stick in PPP/TTY mode, so I can have a public IP4 address on my router. But they do not provide IP6, so the ppp0 interface has only an fe80:* address. Unfortunately it's enough to fool Unbound.

Solution: add

do-ip6: no

under Services -> Unbound DNS -> General -> Custom options.

(For the record: the ZTE MFE 831 USB LTE stick works very well with OPNsense.)

Hello all,

This is my first post here, so a short introduction: I'm a seasoned *nix sysadmin (Solaris and Linux at $DAYJOB, and whatever I can get my hands dirty with at home), but quite new to OPNsense. I tried to look for an answer to my problem, but could not find one - apologies if I just missed something.

I have recently migrated my home network from a Turris Omnia to OPNsense/APU2C2, but DNS resolution with the default Unbound is still flaky. I set it up in standalone mode without upstream resolvers. After restarting Unbound (to clear the cache) the first resolution of any DNS record with the host command is always completely or partially ends in not found/SERVFAIL. The 2nd try works, I assume meanwhile Unbound gets and caches the results.

The clients I tried it with: Macs on WiFi, a wired FreeNAS box (11.2U7) and locally, on the router. Same results.

I used "host" only for testing, but the bug manifests itself with "real" clients, too. And it's pretty annoying when the browser comes back with a "$YOURFAVOURITESEARCHENGINE does not exist". :)

I did not test caching intentionally, but I remember that retrying a DNS record after sufficient time repeated these results: first answer failed, as if Unbound had been restarted.

Alternatively I stopped Unbound and quickly set up DNSmasq with 1.1.1.1 as upstream - it works flawlessly. But I prefer to use my own DNS resolver and not relying on any provider.


Has anyone else experienced the same problem or am I doing something wrong?


(I have been using Unbound on a small and old Debian box for years, but haven't yet seen anything like this.)