[CALL FOR TESTING] Defer PPPoE dialup to after boot completes

[franco]

Hi,

Another thing we're looking at at the moment is the topic of unreliable dialling in PPPoE (although this also affects other PPP the main bulk of users is on PPPoE and it works essentially the same).

This was also discussed in the following topic although multiple problem areas exist around the current integration: https://forum.opnsense.org/index.php?topic=52824.0

The patch itself applies cleanly on top of 26.7.9 and simply moves the dailup to after boot sequence fully completed.  The idea here is to address things like avoiding early hardware quirks during interface init, providing a cleaner framework for CARP support, avoiding DHCPv6 client loops, etc.

# opnsense-patch https://github.com/opnsense/core/commit/39cca4ca785

It needs a reboot to take effect.  If it's not working simply run the patch command again.

All feedback on this is welcome.


Cheers,
Franco

[RamSense]

Hi Franco,

I tested the patch on OPNsense 26.7.9_1 (amd64) and can confirm a successful regression test after reboot.

reboot results:
- Deferred PPP startup confirmed in the system log
- PPPoE connected automatically
- IPv4 default route via PPPoE restored
- IPv4 and IPv6 HTTPS connectivity successful
- No pending PPP startup tasks
- WireGuard interface active and working.

Boot log analysis:
- PPP startup deferred: 1
- PPPoE connection successful: 1
- PPPoE disconnections: 0
- DHCPv6 restarts: 1
- DHCPv6 transmission errors: 0
- Interface startup errors: 0

The PPPoE connection was already working before applying the patch, so this confirms successful startup and no observed regressions in the tested components, rather than that I needed a fix for an existing connection failure.

Thanks for your work on OPNsense!

[abulafia]

Same here; no problems before, no problems after applying the patch and rebooting.

Edit: igb (Intel i350-T4)

System log shows a few errors, which are probably benign (excerpt):

Code Select Expand

[...newer entries show DHCPv6 obtaining an IPv6 PD delegation ...]
026-11-13T18:31:14 Notice dhcp6c send solicit to ff02::1:2%pppoe0
2026-11-13T18:31:14 Notice dhcp6c Sending Solicit on pppoe0
2026-11-13T18:31:14 Notice kernel <118>[39] setup pppoe0 [egress only]
2026-11-13T18:31:14 Notice dhcp6c reset a timer on pppoe0, state=INIT, timeo=0, retrans=275
2026-11-13T18:31:14 Notice dhcp6c failed to get default IF ID for pppoe0, using random ID
2026-11-13T18:31:14 Notice dhcp6c link layer address is too short (pppoe0)
[...newer entries show PPPoE coming up ...]
2026-11-13T18:31:11 Notice ppp process 21112 started, version 5.9
2026-11-13T18:31:11 Notice ppp
2026-11-13T18:31:11 Notice ppp Multi-link PPP daemon for FreeBSD
2026-11-13T18:31:07 Error opnsense /usr/local/etc/rc.bootup: The command `/usr/local/sbin/ifinfo 'pppoe0'' failed to execute
2026-11-13T18:31:07 Error opnsense /usr/local/etc/rc.bootup: The command `/usr/local/sbin/ifinfo 'pppoe0'' failed to execute
2026-11-13T18:31:06 Error opnsense /usr/local/etc/rc.bootup: The command `/usr/local/sbin/ifinfo 'pppoe0'' failed to execute
2026-11-13T18:31:06 Error opnsense /usr/local/etc/rc.bootup: The command `/usr/local/sbin/ifinfo 'pppoe0'' failed to execute
2026-11-13T18:31:06 Warning opnsense /usr/local/etc/rc.bootup: ROUTING: refusing to set interface route on addressless wan(pppoe0)
2026-11-13T18:31:03 Notice opnsense /usr/local/etc/rc.bootup: PPP start deferred during boot on 'pppoe0'
2026-11-13T18:31:02 Notice kernel <118>[27] Configuring VLAN_PPPoE interface...done.
2026-11-13T18:31:02 Notice opnsense /usr/local/etc/rc.bootup: Device vlan0.40 required by VLAN_PPPoE(opt3), configuring now

[hharry]

I've done some testing, OPNsense 26.7.9_1-amd64 + the patch, running in ESXi 6.7 VM, and all looks good

1. confirmed the defered start of mpd5

Code Select Expand

<13>1 2026-11-14T11:29:39+11:00 OPNsense_LAB.localdomain opnsense 596 - [meta sequenceId="303"] /usr/local/etc/rc.bootup: PPP start deferred during boot on 'pppoe1'
<13>1 2026-11-14T11:29:39+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="309"] <6>[48] vmx2: link state changed to UP
<13>1 2026-11-14T11:29:39+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="312"] <6>[48] vmx3: link state changed to UP
<13>1 2026-11-14T11:29:39+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="315"] <6>[48] vmx0: link state changed to UP
<13>1 2026-11-14T11:29:39+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="318"] <6>[48] vmx1: link state changed to UP
<13>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="372"] <118>[53] Reconfiguring IPv4 on vmx2
<13>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="373"] <118>[53] Dialing PPP on wan
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="374"] Multi-link PPP daemon for FreeBSD
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="375"]
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="376"] process 34295 started, version 5.9
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="377"] web: web is not running
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="378"] [wan] Bundle: Interface ng0 created
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="379"] [wan_link0] Link: OPEN event
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="380"] [wan_link0] LCP: Open event
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="381"] [wan_link0] LCP: state change Initial --> Starting
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="382"] [wan_link0] LCP: LayerStart
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="385"] [wan_link0] PPPoE: Connecting to ''
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="386"] PPPoE: rec'd ACNAME "VYOS03"
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="387"] [wan_link0] rec'd proto LCP while dead
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="388"] [wan_link0] PPPoE: connection successful
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="389"] [wan_link0] Link: UP event
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="390"] [wan_link0] LCP: Up event
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="391"] [wan_link0] LCP: state change Starting --> Req-Sent
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="392"] [wan_link0] LCP: SendConfigReq #1
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="393"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:45+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="394"] [wan_link0]   MAGICNUM 0x8fc2e45a
<13>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain kernel - - [meta sequenceId="395"] <6>[54] ng0: changing name to 'pppoe1'
<29>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="396"] [wan_link0] LCP: rec'd Configure Ack #1 (Req-Sent)
<29>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="397"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="398"] [wan_link0]   MAGICNUM 0x8fc2e45a
<29>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="399"] [wan_link0] LCP: state change Req-Sent --> Ack-Rcvd
<13>1 2026-11-14T11:29:46+11:00 OPNsense_LAB.localdomain opnsense 29837 - [meta sequenceId="400"] /usr/local/etc/rc.newwanip: ROUTING: ignoring down gateways: AU761NORDVPNCOM_VPNV4, WAN_PPPOE
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="416"] [wan_link0] LCP: state change Ack-Rcvd --> Req-Sent
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="417"] [wan_link0] LCP: SendConfigReq #2
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="418"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="419"] [wan_link0]   MAGICNUM 0x8fc2e45a
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="420"] [wan_link0] LCP: rec'd Configure Ack #2 (Req-Sent)
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="421"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="422"] [wan_link0]   MAGICNUM 0x8fc2e45a
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="423"] [wan_link0] LCP: state change Req-Sent --> Ack-Rcvd
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="424"] [wan_link0] LCP: rec'd Configure Request #89 (Ack-Rcvd)
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="425"] [wan_link0]   AUTHPROTO CHAP MD5
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="426"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="427"] [wan_link0]   MAGICNUM 0x17ea5805
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="428"] [wan_link0] LCP: SendConfigAck #89
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="429"] [wan_link0]   AUTHPROTO CHAP MD5
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="430"] [wan_link0]   MRU 1492
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="431"] [wan_link0]   MAGICNUM 0x17ea5805
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="432"] [wan_link0] LCP: state change Ack-Rcvd --> Opened
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="433"] [wan_link0] LCP: auth: peer wants CHAP, I want nothing
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="434"] [wan_link0] LCP: LayerUp
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="435"] [wan_link0] CHAP: rec'd CHALLENGE #1 len: 30
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="436"] [wan_link0]   Name: "accel-ppp"
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="437"] [wan_link0] CHAP: Using authname "username@example.com"
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="438"] [wan_link0] CHAP: sending RESPONSE #1 len: 50
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="439"] [wan_link0] CHAP: rec'd SUCCESS #1 len: 28
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="440"] [wan_link0]   MESG: Authentication succeeded
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="441"] [wan_link0] LCP: authorization successful
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="442"] [wan_link0] Link: Matched action 'bundle "wan" ""'
<29>1 2026-11-14T11:29:48+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="443"] [wan_link0] Link: Join bundle "wan"
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="444"] [wan] Bundle: Status update: up 1 link, total bandwidth 64000 bps
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="445"] [wan] IPCP: Open event
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="446"] [wan] IPCP: state change Initial --> Starting
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="447"] [wan] IPCP: LayerStart
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="448"] [wan] IPCP: Up event
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="449"] [wan] IPCP: state change Starting --> Req-Sent
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="450"] [wan] IPCP: SendConfigReq #1
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="451"] [wan]   IPADDR 0.0.0.0
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="452"] [wan] IPCP: rec'd Configure Request #6 (Req-Sent)
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="453"] [wan]   IPADDR 192.168.40.254
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="454"] [wan]     192.168.40.254 is OK
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="455"] [wan] IPCP: SendConfigAck #6
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="456"] [wan]   IPADDR 192.168.40.254
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="457"] [wan] IPCP: state change Req-Sent --> Ack-Sent
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="458"] [wan] IPCP: rec'd Configure Nak #1 (Ack-Sent)
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="459"] [wan]   IPADDR 192.168.40.1
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="460"] [wan]     192.168.40.1 is OK
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="461"] [wan] IPCP: SendConfigReq #2
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="462"] [wan]   IPADDR 192.168.40.1
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="463"] [wan] IPCP: rec'd Configure Ack #2 (Ack-Sent)
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="464"] [wan]   IPADDR 192.168.40.1
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="465"] [wan] IPCP: state change Ack-Sent --> Opened
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="466"] [wan] IPCP: LayerUp
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="467"] [wan]   192.168.40.1 -> 192.168.40.254
<13>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 83722 - [meta sequenceId="469"] ppp-linkup: executing on pppoe1 for inet
<13>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain opnsense 85347 - [meta sequenceId="470"] /usr/local/opnsense/scripts/interfaces/ppp-ipv6.php: ROUTING: entering configure using wan
<13>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain opnsense 85347 - [meta sequenceId="472"] /usr/local/opnsense/scripts/interfaces/ppp-ipv6.php: ROUTING: ignoring down gateways: WAN_PPPOE, AU761NORDVPNCOM_VPNV4
<13>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain opnsense 85347 - [meta sequenceId="473"] /usr/local/opnsense/scripts/interfaces/ppp-ipv6.php: plugins_configure monitor (,[])
<13>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain opnsense 85347 - [meta sequenceId="474"] /usr/local/opnsense/scripts/interfaces/ppp-ipv6.php: plugins_configure monitor (execute task : dpinger_configure_do(,[]))
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="475"] [wan] IFACE: Up event
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="476"] [wan] IFACE: Rename interface ng0 to pppoe1
<29>1 2026-11-14T11:29:49+11:00 OPNsense_LAB.localdomain ppp 34295 - [meta sequenceId="477"] [wan_link0] rec'd unexpected protocol IPv6, rejecting
<29>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain ppp 14387 - [meta sequenceId="478"] Last message '[wan_link0] rec'd un' repeated 1 times, suppressed by syslog-ng on OPNsense_LAB.localdomain
<13>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain opnsense 87249 - [meta sequenceId="479"] /usr/local/etc/rc.newwanip: IP renewal starting (new: 192.168.40.1, old: , interface: wan, device: pppoe1, force: yes)
<13>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain opnsense 87249 - [meta sequenceId="482"] /usr/local/etc/rc.newwanip: ROUTING: ignoring down gateways: WAN_PPPOE, AU761NORDVPNCOM_VPNV4
<13>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain opnsense 87249 - [meta sequenceId="483"] /usr/local/etc/rc.newwanip: plugins_configure monitor (,[WAN_PPPOE])
<13>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain opnsense 87249 - [meta sequenceId="484"] /usr/local/etc/rc.newwanip: plugins_configure monitor (execute task : dpinger_configure_do(,[WAN_PPPOE]))
<13>1 2026-11-14T11:29:50+11:00 OPNsense_LAB.localdomain opnsense 87249 - [meta sequenceId="485"] /usr/local/etc/rc.newwanip: Chose to bind WAN_PPPOE on 192.168.40.1 since we could not find a proper match.

I also did some extra testing and confirm that;

1. PPPoE adaptive LCP echo request is still successfully working as before the patch
2. PPPoE recovers successfully after WAN link failure
3. frr running eBGP automatically re-established BGP session

[RedVortex]

Works on an HA pair (two nodes, CARP, PPPoE on a VLAN) on 26.7.9_1 with 39cca4ca785 plus the CARP check from PR #9896 on top:

Code Select Expand

opnsense-patch 39cca4ca785 f2c74235ed Both nodes log "PPP start deferred during boot", and every test ended with exactly one PPPoE session across the pair.

For HA the deferral needs that check on top. On its own, the deferred start, a reconfigure or a Save/Apply would still dial on the BACKUP. With both, the BACKUP skips the start ("no parent device is CARP MASTER") and only the MASTER dials. Reboots, maintenance mode, failover and failback all behave. The deferral also took over the boot-race handling the PR used to carry, so the PR is now just the CARP check. Thanks Franco.

Details and the 8 test cases: https://github.com/opnsense/core/pull/9896#issuecomment-6871223343

Thanks a lot for this ! :-)