Hi
Had after updating to 20.7.3 on both opnsenses in the Lobby an "unread message" (see attached example), which I don't understand and I have no idea how to follow-up on these...
In the example em2 is the WAN interface and the redacted IP is the ISPs gateway iirc.
Is this something to worry about? How to look at that in detail?
Many thanks in advance!
chemlud
can you please share a full message?
what is in em2 config? (ipv4\6 enabled? dhcp?)
Where to find the full message? :-)
Both WANs are DHCP, ipv4 only...
just the beginning of the sentence is not visible in the screenshot. probably this is "no routing address with matching address family found", but who knows..
but the traffic goes?
Do messages appear on every pf reload or only when the wan address is updated?
can you share some logs before and after that strings
I checked the logs available from GUI, nothing in it, but as this "notice" has no time stamp, hard to know where to look for...
Both WAN IPs are not renewed on a regular base.
Where does this "unread notice" come from? email for root? no idea where to look...
QuoteWhere does this "unread notice" come from?
/tmp/notices i think
if an error occurs while loading rules in pf, error content added to the file.
and php reads it
Nope, there is no such thing as /tmp/notices and all the other files in /tmp are not helpful...
Any ideas?
I found this:
https://github.com/opnsense/core/blob/master/src/www/guiconfig.inc
but no idea where "get_notices" reads from...
Maybe unbound wants to use ipv6 (although not configured by me, but opnsense itself might not care iirc), but it's disabled by me (ipv6 in total and ipv6 for unbound).
But I can't find a related message in System -> Log files -> Backend or General... nor in the unbound log (which looks rather patchy, with some messages for some days in the past (11. 15. 20. 26. 28. September) and then silence for the rest of the days... :-(
QuoteNope, there is no such thing as /tmp/notices and all the other files in /tmp are not helpful
you probably acknowledge all notices and there were no more similar errors. just wait for the next one )
...did a reboot on one sense, but no "notice"... wait'n see...
so good. there may be a one-time error during the update
Got it!
a:1:{i:1601539191;a:1:{s:6:"notice";s:341:"There were error(s) loading the rules: /tmp/rules.debug:186: no routing address with matching address family found. - The line in question reads [186]: pass out route-to ( em2 xx.yy.zzz.aaa ) from {em2} to {!(em2:network)} keep state allow-opts label "470b24148e83cbf020300f9a54691951" # let out anything from firewall host itself (force gw)
";}}
so. this message should be in the main log too. what happened before that? what made the pf rules reload?
gateway swtich? wan address changed? can you share part of log before and after error string?
Is there a time stamp or something like that in the message to make it easier to find the respective part of the log? :-)
If the time is 16:01 the message in the log is
reload filter for configured schedules
QuoteIs there a time stamp or something like that in the message to make it easier to find
i think "{i:1601539191" is epoch timestamp. its
GMT: Thursday, 1 October 2020 г., 7:59:51
also there is a filter in log page.
As I did a reboot at 10 am this morning without warning afterwards the timestamp hypothesis is not plausible to me ;-)
16:01 would be more sensible?
OK you were right, but it's in UTC, so the reboot was at 09:54 and at 09:59 the problem was logged:
2020-10-01T09:59:51 kernel ovpnc2: link state changed to DOWN
2020-10-01T09:59:51 opnsense[65119]
2020-10-01T09:59:51 opnsense[65119] /usr/local/etc/rc.filter_configure: There were error(s) loading the rules: /tmp/rules.debug:186: no routing address with matching address family found. - The line in question reads [186]: pass out route-to ( em2 xxx.yyy.zzz.aaa ) from {em2} to {!(em2:network)} keep state allow-opts label "470b24148e83cbf020300f9a54691951" # let out anything from firewall host itself (force gw)
2020-10-01T09:59:50 opnsense[17765] /usr/local/etc/rc.linkup: Clearing states for stale wan route on em2
2020-10-01T09:59:50 opnsense[17765] /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
2020-10-01T09:59:50 opnsense[33522] /usr/local/etc/rc.newwanip: OpenVPN client 1 instance started on PID 53905.
2020-10-01T09:59:49 opnsense[24279] plugins_configure newwanip (execute task : webgui_configure_do(,lan))
2020-10-01T09:59:49 opnsense[24279] plugins_configure newwanip (execute task : vxlan_configure_interface())
2020-10-01T09:59:49 kernel pflog0: promiscuous mode enabled
2020-10-01T09:59:49 kernel pflog0: promiscuous mode disabled
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (execute task : unbound_configure_do(,lan))
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (execute task : openssh_configure_do(,lan))
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (execute task : opendns_configure_do())
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (execute task : ntpd_configure_defer())
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (execute task : dyndns_configure_do(,lan))
2020-10-01T09:59:48 opnsense[24279] plugins_configure newwanip (,lan)
No idea why the WAN interface detached. Please not that the WAN IP DID NOT CHANGE (it didn't change the last years, although it's DHCP)...
QuoteOK you were right, but it's in UTC
yes. and i selected
GMT for you )
QuoteNo idea why the WAN interface detached. Please not that the WAN IP DID NOT CHANGE (it didn't change the last years, although it's DHCP)...
but OS thinks that cable was unplugged (DEVD Ethernet detached event for wan)
if you can share bigger part of log - may be it Cycling up\down some time?
can try to change speed\duplex and check results
As I wrote, I saw this first time with 20.7.1, before doing fine for more than a year.
Speed/duplex set to default, auto...
Nothing remarkable in the log, but I don't want to redact the whole boot log now to make it available publicly...
PS: this promiscous mode stuff is popping up repeatedly...
https://forum.opnsense.org/index.php?topic=19342.msg89148#msg89148
Quotethis promiscous mode stuff is popping up repeatedly..
kernel pflog0: promiscuous mode enabledits just pf logging interface reloads when pf reloads
if you schedule pf reload every 15 min then "pflog0: " messages every 15 min is good
cycling link up\down - thats not good
Yepp, this "promiscous" thing at 01, 16, 31, and 46 every hour is somewhat normal, but as you can read in the linked thread, it happens more often and sometimes kills my states
PS: Have the identical error with identical log entries (DEVD Ethernet detached event for wan) on a second install after rebooting yesterday evening...
Quoteyou can read in the linked thread, it happens more often and sometimes kills my states
yeah. But i don't think it's about the same. frequent pf reload is probably because of cron jobs. something calls rc.filter_configure too often.
but Ethernet detached event and pfctl load errors has nothing to do with it
something with driver\cable\duplex\speed\upstream switch possibly
Yeah, but
\cable\duplex\speed\upstream
is not plausible, as the two boxes showing identical "notice" have different cable\duplex\speed\upstream, only thing in common is identical hardware (WAN interface is mounted on mobo of Dell Optiplex), so what is most likely is
driver
?
Quote?
I do not know. just guessing. can you try setting the speed and duplex manually to low values?
On both WAN is
Media 100baseTX <full-duplex>
and actually bandwidth is more than 10 Mbit/s and equal or less than 100 Mbit/s, so what?
this is not about the speed of your ISP, but about speed\duplex auto-negotiation between OPNsense and upstream router\switch.
perhaps at some point they start to renegotiate the speed\duplex and the link is down for a short time
And why should that work for more than a year and then start to make problems after an update of the opnsense?
Because a large part of the network interface drivers were rewritten for FreeBSD 12 and iflib. I have occasional glitches with speed/duplex settings, too. But that is something hard to fix for the OPNsense project. Needs to be resolved upstream.
OK, no problem, I'm here to provide details, cause "doesn't work upon reboot" is not really helpful, I guess ;-)
Tried a newer Optiplex (backup machine, all interface emX), no problems on 20.7.1 and 20.7.3 on boot...