Messages

I've been reading in the 16.7 forum how the original Intel driver is solving some problems for people who have been using the standard FreeBSD driver.  Would the "pkg install intel-em-kmod" fix also install on an A10 QC running 16.1.20?

16.1.19 changelog:
...
o interfaces: traffic graphs now show more device types
...

Solved.  Thank you!

Probably way beyond what you want to do, but if you control DNS you can also populate that with Bonjour records.  In my experience it doesn't work for AirPlay, but AirPrint does fall back to regular unicast DNS lookups to discover and browse printers.  Pro:  avoids all the multicast propagation; gives you centralized control over the printer name/location fields.  Con:  a bit of work to set up; your DNS may not allow dynamic updates which will prevent new printers being plugged in from automatically appearing to your users (this is actually a Pro for me).

An ugly-page quick guide:
http://www.dns-sd.org/ServerStaticSetup.html

More info at:
https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/NetServices/Introduction.html

Both.  Those interfaces are still being tracked in Health, but are absent from the real-time Traffic Graph page and Dashboard widget.  Truthfully, I'd never used the widget before now so I can't say whether this is a change.  But they were definitely selectable and useable in the old T.G. page.

I updated a box with PPPoE WAN connections to the 16.1.18 from .14 or so, and noticed the new Traffic Graph re-work for the first time.  The PPPoE interfaces are not included in the graphs, although they are listed in the drop-down box underneath.  The statically assigned or DHCP assigned interfaces are included.

Is this a bug, or something odd in my config, or is there something I am supposed do to get them to appear in the graph?  On the other side of the coin, there are some interfaces I have no need to view (e.g. IPSec) and would like them to exclude them.  Generally, do I have any control over what appears in the Traffic Graphs?

The issue has not recurred with the static address version of my config since my update of May 3, so maybe something else was the problem that day.

I'm now able to put the dynamic PPPoE version of my config back into production for a while to see if the issue still persists for that setup.  The box is running the latest 16.1.18.  If it does happen again, I'll report here.

Thanks for checking back on this!

A follow-up on this issue.  I reverted to another OPNsense config where the WAN links are just static addresses instead of dynamic PPPoE connections.  The PPPoE (and NAT) is then handled by border routers cabled directly to the OPNsense.  This worked well for months with 15.7, so I hoped it would clear up the issue.

Unfortunately one of the OPNsense WAN links went down again.  The border router attached to the link was fine, still PPPoE connected and functioning properly.  But the OPNsense could not ping or connect to the router.  I tried swapping that WAN cable to another border router, still OPNsense couldn't ping/connect to it.

system.log had a few hundred lines over a few seconds of:
May  3 09:12:23 OPNsense kernel: arpresolve: can't allocate llinfo for XXX.XXX.XXX.XXX on em2
The XXX's being the static IP of that particular OPNsense WAN interface.

Tried bartjsmit's suggestion of Disable / Enable the interfaces.  It did not help, but the logs show OPNsense running through another few hundred 'arpresolve' system.log lines again after I Enabled the interface.  I also tried manually 'ifconfig em2 down' and 'ifconfig em2 up', no help either.

So, I don't think this issue rests on PPPoE.  This box has no PPPoE config anywhere in it.  But the general symptom of an interface suddenly going deaf/mute, where no prodding will get it going again until a reboot, that seems similar.

I have two OPNsense boxes and have experienced issues on both now with 16.1 that I never had with 15.7.  I can't be sure it's not a hardware fault, but is there an easy way to downgrade back to 15.7 to see if the problems follow me?  Or, preferably, any better troubleshooting suggestions?

I can't remember if I've done that or not.  I know Disconnect / Connect the PPPoE connection didn't work.  I'll give your interface Disable / Enable tip a try next time.  Thanks.

I have rebooted.  No improvement, although the service status shows apinger is running.  The gateways.log now shows:

Apr 22 22:25:24 OPNsense apinger: Starting Alarm Pinger, apinger(5361)
Apr 22 22:25:24 OPNsense apinger: No usable targets found, sleeping...
Apr 22 22:25:25 OPNsense apinger: No usable targets found, sleeping...
Apr 22 22:26:23 OPNsense apinger: No usable targets found, sleeping...


I tried clicking on the Stop Service and Restart Service buttons for apinger but they didn't seem to respond and the log kept on growing as above.  From the console I tried a '/usr/local/etc/rc.d/apinger stop' and it sat at 'Waiting for PIDS:' longer than my patience, so I just 'kill -KILL'ed the PID.  The status now showed Stopped, and clicking the Start Service button worked.  Then apinger was working properly and my gateway status updating properly.

For what it's worth this happened with both the _6 and _7 packages.

It happened again, so I copied off everything from /var/log before rebooting.  Looking over the files, I really don't see anything in the time leading up to the disconnect except the ppps.log.  I've only scrubbed out the IP address from the "Delete route" logline.

Apr 22 22:00:10 OPNsense ppp: [opt2_link0] LCP: no reply to 1 echo request(s)
Apr 22 22:00:20 OPNsense ppp: [opt2_link0] LCP: no reply to 2 echo request(s)
Apr 22 22:00:30 OPNsense ppp: [opt2_link0] LCP: no reply to 3 echo request(s)
Apr 22 22:00:40 OPNsense ppp: [opt2_link0] LCP: no reply to 4 echo request(s)
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] LCP: no reply to 5 echo request(s)
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] LCP: peer not responding to echo requests
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] LCP: state change Opened --> Stopping
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] Link: Leave bundle "opt2"
Apr 22 22:00:50 OPNsense ppp: [opt2] Bundle: Status update: up 0 links, total bandwidth 9600 bps
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: Close event
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: state change Opened --> Closing
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: SendTerminateReq #4
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: LayerDown
Apr 22 22:00:50 OPNsense ppp: [opt2] IFACE: Delete route 0.0.0.0/0 XXX.XXX.XXX.XXX failed: No such process
Apr 22 22:00:50 OPNsense ppp: [opt2] IFACE: Down event
Apr 22 22:00:50 OPNsense ppp: [opt2] IFACE: Rename interface pppoe1 to pppoe1
Apr 22 22:00:50 OPNsense ppp: [opt2] IPV6CP: Close event
Apr 22 22:00:50 OPNsense ppp: [opt2] IPV6CP: state change Stopped --> Closed
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: Down event
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: LayerFinish
Apr 22 22:00:50 OPNsense ppp: [opt2] Bundle: No NCPs left. Closing links...
Apr 22 22:00:50 OPNsense ppp: [opt2] IPCP: state change Closing --> Initial
Apr 22 22:00:50 OPNsense ppp: [opt2] IPV6CP: Down event
Apr 22 22:00:50 OPNsense ppp: [opt2] IPV6CP: state change Closed --> Initial
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] LCP: SendTerminateReq #2
Apr 22 22:00:50 OPNsense ppp: [opt2_link0] LCP: LayerDown
Apr 22 22:00:52 OPNsense ppp: [opt2_link0] LCP: SendTerminateReq #3
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] LCP: state change Stopping --> Stopped
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] LCP: LayerFinish
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] PPPoE: connection closed
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] Link: DOWN event
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] LCP: Down event
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] LCP: state change Stopped --> Starting
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] LCP: LayerStart
Apr 22 22:00:54 OPNsense ppp: [opt2_link0] Link: reconnection attempt 1 in 1 seconds
Apr 22 22:00:55 OPNsense ppp: [opt2_link0] Link: reconnection attempt 1
Apr 22 22:00:55 OPNsense ppp: [opt2_link0] PPPoE: Connecting to '1'
Apr 22 22:01:04 OPNsense ppp: [opt2_link0] PPPoE connection timeout after 9 seconds
Apr 22 22:01:04 OPNsense ppp: [opt2_link0] Link: DOWN event
Apr 22 22:01:04 OPNsense ppp: [opt2_link0] LCP: Down event
Apr 22 22:01:04 OPNsense ppp: [opt2_link0] Link: reconnection attempt 2 in 4 seconds
Apr 22 22:01:08 OPNsense ppp: [opt2_link0] Link: reconnection attempt 2
Apr 22 22:01:08 OPNsense ppp: [opt2_link0] PPPoE: Connecting to '1'
Apr 22 22:01:17 OPNsense ppp: [opt2_link0] PPPoE connection timeout after 9 seconds
Apr 22 22:01:17 OPNsense ppp: [opt2_link0] Link: DOWN event
Apr 22 22:01:17 OPNsense ppp: [opt2_link0] LCP: Down event
Apr 22 22:01:17 OPNsense ppp: [opt2_link0] Link: reconnection attempt 3 in 3 seconds


...repeat repeat repeat.  As before, rebooting OPNsense brought the connection up again.

Thanks for the fix.  I've applied it and will let you know after next reboot.

My opnsense 16.1 router has exhibited two problems in recent weeks, neither of which appeared during several months of use with 15.7.  Both seem related to the fact that my internet links are via PPPoE.

The connection down problem is pretty severe and my only solution so far is to reboot:

One of my PPPoE connections will go down.  OPNsense will try to reconnect but timeout after 9 seconds and retry, over and over and over, thousands of times, probably forever if I don't intervene.  My first thought was that something upstream had failed, but this has happened several times now and I've had some opportunities to fiddle around during breakage.

If I move the "connection down" cable from the OPNsense to a laptop and configure the same PPPoE connection it comes right up.  I can also move that cable to the router I used prior to deploying OPNsense and it will also bring the same PPPoE connection up.  But even moving the cable back to the OPNsense at this point, it will only retry retry retry.  Rebooting the OPNsense clears it's head however, and it can then successfully bring up the PPPoE connection as if nothing was ever wrong.

The symptoms sound the same as https://forum.opnsense.org/index.php?topic=2337.0 but that thread seems to show that the problem was resolved with 16.1.5.  I started the 16.1 series from 16.1.7.  There's some mention of RFC 4638 support having been implemented but my PPPoE connection MTU config in OPNsense is the ISP-recommended 1454, and should have nothing whatsoever to do with any RFC 4638 code.  No VLANs are used, the OPNsense is cabled directly to the ISP equipment.

Any help appreciated.  What other info could I provide that would help you help me?

My opnsense 16.1 router has exhibited two problems in recent weeks, neither of which appeared during several months of use with 15.7.  Both seem related to the fact that my internet links are via PPPoE.

The apinger problem is mostly an annoyance, and I have a work-around:

After booting up, the apinger service has failed to start, and my gateway statuses only show "Pending..." forever.  Looking at the gateways.log shows e.g.
Apr 18 08:51:55 OPNsense apinger: Starting Alarm Pinger, apinger(6114)
Apr 18 08:51:55 OPNsense apinger: No usable targets found, exiting

I'm guessing the PPPoE connections haven't been made yet, so apinger figures there's nothing to do.  By the time I log into the web gui moments later, however, the PPPoE connections have come up, the IP addresses are shown in the Dashboard etc, and traffic is flowing nicely.  But apinger never seems to try again, I've tried waiting for hours.  My workaround is to log into the web gui and hit the Start Service button on the Gateways page.  Is there something I can do to fix this?  Either have apinger not bail out so fast, or retry again periodically, or something?

I bought the Quad Core version for my workplace, beefier than the Dual Core version but maybe my experience with it is helpful.  It has replaced more expensive competitor products as a multi-wan border router.  I learned about OPNsense from a post on PipeDot (http://pipedot.org/story/2015-02-17/end-of-the-m0n0wall-project).  When looking to replace some equipment that was approaching End-of-Support, I chose the A10 as I didn't want to be tweaking hardware while also figuring out how to configure OPNsense as an exact drop-in replacement for the EoS equipment.  I also hoped that some of the money spent on the A10 would make it's way over to the OPNsense side of the business.

Things I like about my A10:

• Price.  There are cheaper boxes, but the power-to-price ratio on the A10 is quite refreshing.
• Power (1).  My A10 handles a typical work day of constant 225~275Mbps throughput across 2,000 device clients, load average hovers around 1.0, Dashboard CPU usage hovers around 15%.
• Power (2).  20 Watts?  Nice.  Website for the Dual Core says 15 Watts.

So, for what I need it to do, my A10 handles it easily and I know I could fold more work into it should the need arise.  e.g. I already have an excellent traffic shaper, but were it to fail I would not hesitate to dummynet my A10 until it got fixed.

Things I don't like about my A10:

• Edit: Thanks for the head-smack, jschellevis!  I was wrong about this.  Console Port.  Their Gen3 appliances have a real serial console but the A10 series have this serial-to-USB thing that requires the client OS to have drivers.  Ironically, I cannot find drivers for FreeBSD.  If you never use the console port or use a driver-supported client OS, this won't bother you.
• External or Remote Power Control.  You can power off the A10 from inside OPNsense, but once done the only way to turn it on again is to pull the power cord and re-plug it.  No problem if it's on your desk, inconvenient if it's crammed in a rack.
• No Beep.  OPNsense can beep the PC oscillator upon successful startup/shutdown (System / Settings / Notifications) but either the A10 has no such speaker or it's too quiet for me to hear.  Some kind of clue that it's up would be nice, preferably a status LED.  There is a power LED and if that's off, you know the A10 is off.

That list of dislikes is not a particularly scary one.  The console port thing is the biggest problem for me, the others are mere inconveniences/nitpicks.

I don't have much experience with OPNsense yet, but I do have a box doing multi-wan across two gateways (on OPT1 and OPT2) via Firewall rules.  Works great for all clients passing through the box from LAN, but the box itself initially could not ntp sync or do package updates.  Eventually surmised that box-generated traffic was directed to the Gateway that was marked as Default Gateway (WAN, disconnected in my case).  Duh.  I marked one of my OPT Gateways as Default and box-generated traffic has worked ever since.

Maybe your default gateway is not set, or set to the wrong gateway?