Very unstable PPPoE Connection for WAN

[arnebeh]

Hi all!

I am trying to run an OPNsense firewall on our site connected via an Ethernet Switch and PPPoE. But the connection drops after a few minutes and is not reastablished by the firewall.

Problem description:
- Hardware is a OPNsense DEC2750
- Running OPNsense  22.10-amd64
- We are connecting via Ethernet to a switch in the same rack (which is connected using fibre) on Port 1 of the DEC2750
- For connecting to ISP (Wilhem.Tel), PPPoE ist used.
- Connected to an ECS4120-28T Edge-Core swittch.
- PPPoE is very unstable when using the OPNsense HW, PPPoE drops every 5-30 minutes and is (sometimes) recovering with the device shown up in the dashboard, but no external IP-Adress is established. It also seems just to restart the multilink ppp daemon without any known reason:
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="145"] [wan] IFACE: Rename interface ng0 to pppoe0
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="146"] [wan_link0] rec'd unexpected protocol IPv6
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="147"] [wan_link0] rec'd unexpected protocol IPv6
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="148"] [wan] IPV6CP: rec'd Configure Ack #1 (Ack-Sent)
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="149"] [wan] IPV6CP: state change Ack-Sent --> Opened
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="150"] [wan] IPV6CP: LayerUp
<30>1 2022-11-16T19:27:01+01:00 OPNsense.localdomain ppp 90184 - [meta sequenceId="151"] [wan]   f690:eaff:fe00:959c -> 066c:9dff:fe54:bc13
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="1"] Multi-link PPP daemon for FreeBSD
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="2"] 
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="3"] process 51937 started, version 5.9
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="4"] web: web is not running
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="5"] [wan] Bundle: Interface ng0 created
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="6"] [wan_link0] Link: OPEN event
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="7"] [wan_link0] LCP: Open event
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="8"] [wan_link0] LCP: state change Initial --> Starting
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="9"] [wan_link0] LCP: LayerStart
<30>1 2022-11-16T19:33:25+01:00 OPNsense.localdomain ppp 51937 - [meta sequenceId="10"] [wan_link0] PPPoE: Connecting to ''
<30>1 2022-11-16T19:3


- The same connection is rock stable with an ASUS RT-AX55 Router.

- I also noticed that the LAN-side also seems to be unstable with multiple messages on client laptop side:
[203836.547849] r8152 5-1.1:1.0 enx3ce1a14b1c51: carrier off
[203839.811482] r8152 5-1.1:1.0 enx3ce1a14b1c51: carrier on
[204049.364973] r8152 5-1.1:1.0 enx3ce1a14b1c51: carrier off
[204056.340051] r8152 5-1.1:1.0 enx3ce1a14b1c51: carrier on
This LAN connection is also stable when using the ASUS Router.

Might this be an hardware issue? Or any ideas, what is going on?

Thanks in advance,
Arne

[zyos]

Hi Arne,

I'm very much new to OpnSense but in an effort to try and give back to the community I'll try to help here.

Have you attempted a packet capture on the device to see what it was for you to see? Perhaps there is a driver issue or it's misconfigued in an unexpected way causing an arp storm or something. If this is not yet in production this might be feasible to do and they do tend to "tell all".

Hope you can find something!

-Zyos