[isolated: see #91] PPPoE reconnect loop

[schnipp]

@schnipp: Do you use VLANs for PPPoE? I can't imagine this is a general problem but perhaps something that with a reconnect mpd can't push into VLAN

Yes, my PPPoE connection needs VLAN encapsulation (ID 7). But in my case this is not a task of opnsense as in my setup VLAN encapsulation is done by the DSL modem (Fritzbox 7412). PPPoE forwarding by the DSL modem works flawlessly, only incoming packets of unknown PPPoE sessions will be filtered.

I guess mpd is not aware of vlan tagging since tagging is presumably done in the kernel and seems to be transparent to netgraph nodes connected to the virtual interface.

[schnipp]

[...]
Perhaps you see something blocked in system.log or filter.log. Just a guess since it hit upon this rules which blocks everything and *could* be related to non ipv4/6

I don't know if this makes sense, because I directly tap the netgraph hooks of the mpd daemon.
Does anybody know where exactly firewall filtering takes place in the whole network stack?

[nallar]

No rush, if you can let us know in a week or two. :)


Thanks,
Franco

Still seems to be working :)

[Alphakilo]

I did some more investigation in this topic and increased the logging of the mpd daemon to get some more information of the ISP's LCP echo probing. I found out that the daemon processes echo request packets and itself claims to send out corresponding reply packets. Unfortunately, the echo reply packets are not seen on the WAN interface :-(

Boom:

Code Select Expand

# tcpdump -i re1_vlan7 pppoes and ppp proto 0xc021
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re1_vlan7, link-type EN10MB (Ethernet), capture size 262144 bytes
21:08:29.423673 PPPoE  [ses 0x6cd1] LCP, Echo-Request (0x09), id 228, length 10
21:08:29.423880 PPPoE  [ses 0x6cd1] LCP, Echo-Reply (0x0a), id 228, length 10
21:08:39.423718 PPPoE  [ses 0x6cd1] LCP, Echo-Request (0x09), id 229, length 10
# Saved settings for WAN interface, triggering a reconnect
21:08:39.621649 PPPoE  [ses 0x6cd1] LCP, Term-Request (0x05), id 3, length 6
21:08:43.378083 PPPoE  [ses 0x8758] LCP, Conf-Request (0x01), id 1, length 20
21:08:43.380642 PPPoE  [ses 0x8758] LCP, Conf-Request (0x01), id 1, length 18
21:08:43.390085 PPPoE  [ses 0x8758] LCP, Conf-Reject (0x04), id 1, length 8
21:08:43.390547 PPPoE  [ses 0x8758] LCP, Conf-Request (0x01), id 2, length 16
21:08:43.400199 PPPoE  [ses 0x8758] LCP, Conf-Ack (0x02), id 2, length 16
21:08:45.400987 PPPoE  [ses 0x8758] LCP, Conf-Request (0x01), id 3, length 16
21:08:45.410645 PPPoE  [ses 0x8758] LCP, Conf-Ack (0x02), id 3, length 16
21:08:46.369960 PPPoE  [ses 0x8758] LCP, Conf-Request (0x01), id 2, length 20
21:08:46.370666 PPPoE  [ses 0x8758] LCP, Conf-Ack (0x02), id 2, length 20
21:08:57.028608 PPPoE  [ses 0x8758] LCP, Echo-Request (0x09), id 1, length 10
21:09:07.028322 PPPoE  [ses 0x8758] LCP, Echo-Request (0x09), id 2, length 10
21:09:17.027982 PPPoE  [ses 0x8758] LCP, Echo-Request (0x09), id 3, length 10
21:09:27.032313 PPPoE  [ses 0x8758] LCP, Term-Request (0x05), id 3, length 6

Take note of the identifier (id 3) of the last two packets.
If the endpoint follows RFC1661 that means the Term-Request is a direct consequence of the (unanswered) Echo-Request.

I think I owe you a beer, schnipp.

I have a lot going on on my installation (18.1.5). IDS, VPNs, DHCPv4/6, unbound, ...

[mimugmail]

I'd wonder what happens when you set LCP keepalive to be send every second?
Would this force OPN send Echo Requests instead of just receiving them?

Just an idea ...

[schnipp]

I'd wonder what happens when you set LCP keepalive to be send every second?
Would this force OPN send Echo Requests instead of just receiving them?

Just an idea ...

In my configuration it is set by default (set link keep-alive 10 60). I tried to adjust these values in the past without success. I have not seen any keep-alive packets on wire sent out by the mpd daemon, presumably due to other incoming traffric (see http://mpd.sourceforge.net/doc5/mpd20.html).

[marjohn56]

@schnipp

Is this problem apparent on a 'simple' setup, i.e. no IPS, no VLAN, no VPN etc, just a basic get me to the WAN system? I use pppoe and have zero problems. OK, my system is all statics, but I need pppoe for the initial link up.

[mimugmail]

I'd wonder what happens when you set LCP keepalive to be send every second?
Would this force OPN send Echo Requests instead of just receiving them?

Just an idea ...

In my configuration it is set by default (set link keep-alive 10 60). I tried to adjust these values in the past without success. I have not seen any keep-alive packets on wire sent out by the mpd daemon, presumably due to other incoming traffric (see http://mpd.sourceforge.net/doc5/mpd20.html).

I can add a field for this, but only if it works .. can you try to set it to 1 seconds? I'm not into it but I'd guess if you don't receive in given time the mpd starts sending self, if you set to 5 sec not really sure if this works but with 1 it should ... perhaps worth a try, but it's hard to debug from remote.

[schnipp]

Of course, I can try it.
If somebody can tell me, I can alternatively modify the script/global config file with builds the mpd_wan.conf

[schnipp]

Over a longer period of time I noticed that this issue sometimes also occurs directly after rebooting the machine. So, it is not only related to a pppoe reconnect. Maybe it's a timing issue.

[elektroinside]

This never happened to me...

[schnipp]

I can add a field for this, but only if it works .. can you try to set it to 1 seconds? I'm not into it but I'd guess if you don't receive in given time the mpd starts sending self, if you set to 5 sec not really sure if this works but with 1 it should ... perhaps worth a try, but it's hard to debug from remote.

Can you tell me, where OpnSense stores the initial PPP configuration data which builds mpd_wan.conf file at system bootup?

[franco]

The data is inside /conf/config.xml ... the assembly takes place in /usr/local/etc/inc/interfaces.inc function interface_ppps_configure()


Cheers,
Franco

[mimugmail]

Today I implemented my own OPNsense at home with pppoe and got to the same error. DSL was online and ran into LCP echo timeout.

I did some further investigation and saw that under Interfaces - Assignment the WAN was bound to igb0_vlan7 which is perfectly fine when setting up a DSL, but there is also pppoe_vlan7 available, and after changing to this it's stable now.

No idea if this would fit your problems too but worth a look at assignments.

[franco]

pppoe0_vlan7, this sounds funky and not completely right... should be dead after reboot?