**PPPoE gateway group retains disconnected WAN: stale watcher status and interface-only route-to — workaround tested on 26.7.6**
Hello,
We encountered a reproducible Multi-WAN failure with two PPPoE connections in the same gateway group, both at Tier 1.
Environment:
- OPNsense 26.7.5, subsequently upgraded to 26.7.6; the failure persisted.
- Two IPv4 PPPoE WANs.
- Both gateways monitored using different external IPs.
- Gateway group trigger: Member Down.
- LAN pass rule uses the gateway group.
- Shared forwarding disabled.
- Automatic source NAT rules present for both WANs.
When one PPPoE WAN was taken down and lost its gateway address, LAN clients could no longer access the Internet through the group. Selecting the surviving gateway directly in the LAN rule worked.
The routing table correctly selected the surviving WAN as the default route. However, the generated LAN policy rule still contained both interfaces without gateway addresses:
```text
route-to { pppoe0 , pppoe1 } round-robin sticky-address
```
The loaded PF rule displayed:
```text
route-to { (pppoe0 ?), (pppoe1 ?) } round-robin sticky-address
```
Raw monitor output for the disconnected gateway showed:
```text
status: down
gateway: ""
loss: "~"
```
The surviving gateway reported `none`, a valid gateway address and 0% packet loss.
We found a code path consistent with this behavior:
1. `dpinger_status()` returns `down` with `loss: ~` when monitor data is unavailable.
2. `gateway_watcher.php` skips reports with `loss: ~`, before updating the cached status or triggering a routing alarm.
3. `return_gateways_status()` replaces the raw status with the cached watcher status when `loss` is `~`.
4. A previously cached `none` can therefore make the disconnected gateway appear usable to `getActiveGroups()`.
We did not capture the watcher cache during the original failing test, so that particular cache value remains inferred from the code path.
We applied this local change in `/usr/local/opnsense/scripts/routes/gateway_watcher.php`:
Original:
```php
if ($report['loss'] == '~') {
/* wait for valid data before triggering an alarm */
continue;
}
```
Replacement:
```php
if (
$report['loss'] == '~' &&
!($report['status'] === 'down' && empty($report['gateway']))
) {
/* Wait for data unless the gateway has disappeared. */
continue;
}
```
This allows an explicit `down` report without a gateway address to reach the existing transition, cache and alarm handling. Reports without monitor data but with a gateway address retain the previous waiting behavior.
We backed up the original file, validated PHP syntax and restarted the watcher to load the change.
Afterward, we verified:
- WAN A down: the LAN rule contained only WAN B with its gateway address; HTTPS returned HTTP 200.
- WAN A restored: both gateways returned to the group automatically.
- WAN B down: the LAN rule contained only WAN A; HTTPS returned HTTP 200.
- WAN B restored: both gateways returned to the group automatically.
These tests covered PPPoE disconnection with loss of the gateway address. They do not establish behavior for an upstream outage while the PPPoE session remains established.
Could maintainers review whether this missing-data handling can retain a disconnected PPPoE gateway, and whether the appropriate fix belongs in the watcher or in the cached-status merge?
This is a locally tested workaround, not an official patch.