Hi,
Due to a user prompt we're going to restructure PPP for the next major release 25.1 in order to be able to allow IPv6-only deployments including using DHCPv6 which isn't possible at all at the moment.
A POC already exists and appears to work, but I'd like to take this a bit further due to related issues such as the infamous "Use IPv4 connectivity" setting. The idea here is to remove it and make it the default, because
(a) everyone including developers have trouble wrapping their head around the concept,
(b) when in reality the opt-in setting is really the default behaviour of PPP-based setups to begin with,
(c) and to achieve running DHCPv6 on the hardware interface beneath PPP simply assign the hardware interface and set it to DHCPv6. ;)
The goal is to finish this change on the development version coming with 24.7.2 for general consumption, but it would be very beneficial to merge backwards-compatible changes to 24.7.x without removing "Use IPv4 connectivity" usage as this progresses and matures. The final code change for 25.1 will then be much smaller and easier to review.
And this is where the call for testing takes off at the moment for everyone running 24.7.1:
https://github.com/opnsense/core/commit/3a9f98843b
# opnsense-patch 3a9f98843b
If you have an PPP(oE) deployment with or without IPv6 try this patch and see if it works like before. I'd like to know of behavioural changes or drop in IPv6 connectivity if previously working so we can refine this together, add it to a 24.7.x release and work on the next batch.
Thanks,
Franco
With M-Net, this works (did a reboot after the patch).
IPv4: PPPoE
IPv6: DHCPv6
Prefix-only requested and using the new "Optional Prefix ID" feature.
Nice, thanks. I finished the pieces for opnsense-devel now. Will test this as good as I can and then we'll revisit after 24.7.2 is out.
Just to be sure: you are using "Use IPv4 connectivity", right?
Cheers,
Franco
Applied the patch, seems to be working in my case.
Attached a screenshot of my settings.
Let me know if you need further details or testing done.
@Christophe999s thanks a lot, no further questions at the moment <3
Cheers,
Franco
Quote from: franco on August 09, 2024, 09:49:26 AM
Just to be sure: you are using "Use IPv4 connectivity", right?
Yes.
Just so I didn't misunderstand. Will it still be possible to use the IPv4 connection to get an IPv6 prefix in the future?
Deutsche Telekom continues to work in such a way that primarily an IPv4 connection is established and the IPv6 prefix is requested via the IPv4 connection.
Short answer: yes.
Long answer: Yes, though:
1. "Use IPv4 connectivity" option will be removed while making it the default behaviour.
2. PPP will be used even when IPv4 mode is "None" and IPv6 mode is "DHCPv6" by virtue of being assigned to the interface (device name match). This will be new territory.
It would also be possible, later on, to run DHCP on top of PPPoE, but I don't think this will be needed. The key here is the decoupling of when PPP is started regardless of what IPv4 and IPv6 modes are chosen as long as a relevant mode for PPP is being offered (this is fully backwards compatible).
The only downside in the future is for users of disabled "Use IPv4 connectivity" is that in order to keep IPv6 connectivity they need to assign the parent device of the PPP and configure it as a separate interface.
Cheers,
Franco
Alright. Thank you, franco. This sounds very good!
applied the patch with deutsche telekom (non business) with the checkbox request only prefix turned off and it seems to work normally.
the device came up normally after reboot and connection is established and working as usual.
I've just applied the patch and rebooted without problems.
The only thing that's happened is my download speed (using speedtest) has almost disappeared - this is the result after applying the patch: https://www.speedtest.net/result/16604096563 - although this is typical of the five results today and using various download servers for the test.
I've recently upgraded my service to 900Mbbs, I was on the 500Mbps and always had a download speed was roughly 460 Mbps - this doesn't want to go any faster. I'll have to contact Zen again to see what's happening.
@phoenix: If you are still on that Intel 4560U, this is probably all you can get... ;-)
After updating the firmware to 24.7.1. Connectivity is intermettintly being interrupted. For instance, any SSH session diconnnects every 30 seconds. This happens when DHCPv6 is enabled.
I also had to re-enter my pppoe username and password and re-link the pppoe interface with the vlan, as the pppoe configurations were gone. The interface itself remained, though
Applying the patch didn't change anything. The use ip4 connectivity is still there. I had to disable DHCPv6 on my WAN to solve the interruptions.
My network adapters are intel i226, 3 of them bridged and used for my LAN.
I'm not sure if this is useful for you to troubleshoot, so let me know if any logs would be helpful
I also activated the patch and everything is working without an issue.
IPv4: PPPoE
IPv6: DHCPv6 (with "Use IPv4 connectivity" active)
But I saw there something strange:
It seems that after applying the patch, an additional Gateway appeared.
Background:
To access my DSL modem I configured the interface of the PPPoE connection additionally with a static IP (see attachments).
I'm not 100% sure this appeared with the patch, but I can not remember seeing this interface before applying the patch, therefore I assume the patch is the source of it.
I would expect the patch to misconfigure PPP configuration which would lead to permanent connectivity loss or partial loss because it fails to bring up IPv6 connectivity after connecting the PPP. Everything else looks manual or coincidental. It's easy to revert the patch (run the patch command again) and double-check. Especially things like assigned interfaces are what they are. The patch (or 99.9% of the code in general) does not meddle with the local settings at all.
Cheers,
Franco
Quote from: computeralex92 on August 10, 2024, 10:54:15 AM
But I saw there something strange:
It seems that after applying the patch, an additional Gateway appeared.
Background:
To access my DSL modem I configured the interface of the PPPoE connection additionally with a static IP (see attachments).
No extra gateway here with same config. With the patch applied, everything works as before.
Thanks for the input. Sounds pretty good. Would like to ship this in 24.7.2 with the feedback we have now but in terms of being careful it's probably better to do 24.7.3 for this particular patch. The next iteration will be a bit more invasive in terms of when PPP is started and stopped, but it should also not cause any change in observed behaviour (unless there is a bug we need to catch).
Cheers,
Franco
Thanks a lot testing my strange "behavior".
I removed the patch again and after a reboot, the additional gateway disappeared.
Maybe it is a side effect of having multiple patches active at the same time; I removed them now all:
287c13beb8e interfaces: avoid touching SLAAC address for now
9ddd363bdbb firmware: add upgrade hint to 24.7 crossing ABI
3a9f98843b0 interfaces: move IPv6 connectivity to a separate s
Quote from: computeralex92 on August 11, 2024, 12:07:55 PM
287c13beb8e interfaces: avoid touching SLAAC address for now
Hmmm, but this is not intended for 24.7.1, will apply reversed (i.e., undo the included patch to 24.7 state)¨
A bit late to the party, but I can confirm everything works as before here.
What I've done:
- opnsense-patch
- reboot my box
- reboot my main switch, to assure all devices get new DHCP leases (not necessary, but I upgraded the firmware and took the opportunity --> rebooting the main switch takes down the whole network as the second switch and APs are PoE powered)
My setup (G.fast line)
- Bare metal, WAN port is a dedicated interface. No VLAN, DHCP option or anything on WAN. OPNsense 24.7.1
- IPv4: PPPoE
- IPv6: DHCP6, getting a /48
- In WAN setup -> PPPoE configuration "Use IPv4 connectivity" is marked and prefix size is set to 48, everything else is default.
Everything works. IPv4 only sites, IPv6 only sites, no additional gateway, no speed decrease or anything.
Applied, rebooted, all well with German Telekom business DSL line.
I never used "track interface", though. All static internally.
Tried out the patch & rebooted(pppoe, ipv6 dual stack, /48 prefix dynamically retrieved) - all seems to work fine
I also have applied the patch on my OPNsense and everything is fine. I left activated the IPv6-Feature "Request only prefix" (because many VLANs), meaning i didn't make any changes.
I'm using Opnsense with a Freedom Internet FttH connection in The Netherlands. It seems to work like before.
Thanks all! We can more or less wrap this first batch up. Let's wait for 24.7.3 to hit and we'll see about that second batch.
Cheers,
Franco
Applied patch. Did NOT reboot. Disabled WAN IF, saved, applied. Enabled WAN IF, saved, applied.
All fine, inluding IPv6 /56 prefix.
ISP: Digi Romania.
Thank you :)
Might just hand out the second patch sooner (i.e. later today). Feeling lucky.
Cheers,
Franco
Will this restructure improve speeds (work around 1core issue)
Nope.
While we wait for our traceroute situation to get better here's the second PPPoE patch:
https://github.com/opnsense/core/commit/3a9f98843b <-- old patch foundation
https://github.com/opnsense/core/commit/182bab339a <-- new patch on top
# opnsense-revert opnsense && opnsense-patch 3a9f98843b 182bab339a
With the second patch, the decision to start and stop mpd5 is no longer directly tied to the IPv4 and IPv6 mode set up in the interface, but the whole thing should still be 100% compatible.
Cheers,
Franco
I didn't try the first patch, but have just applied the new one(s). After a reboot, everything seems OK so-far. I get a /56 prefix via DHCPv6 over PPPoE, and have one LAN interface statically configured with a /64, and another tracking the WAN interface.
I just looked at my WAN interface configuration again, and I see that the "Use IPv4 connectivity" option is there. Isn't it supposed to disappear with these patches?
Not just yet.
It's still there indeed. The exercise here is to get these test patches into 24.7.x and keep the breaking change(s) like the flag removal hidden on what is to become 25.1.
Cheers,
Franco
Quote from: franco on August 13, 2024, 12:31:41 PM
# opnsense-revert opnsense && opnsense-patch 3a9f98843b 182bab339a
Done and still working - same setup as described here (https://forum.opnsense.org/index.php?topic=42081.msg207613#msg207613).
Quote from: franco on August 13, 2024, 12:31:41 PM
# opnsense-revert opnsense && opnsense-patch 3a9f98843b 182bab339a
Working the same here too.
I had something odd happen overnight. I don't know if it's related to these patches, or just a coincidence.
I woke up in the middle of the night, and found that I had "no internet". I went to my office and looked at opnsense, which showed the PPPoE interface as red. Not fully awake, I tried to "reload" the interface (from the overview page). It ("the internet") didn't come back (I think the interface was still showing red), so I reboot, and then it did come back, and everything was OK again.
Now that I'm more (but probably still not 100%) awake, I'm looking in /var/log/ppps. It appears that there was some sort of outage lasting about 12 minutes (assuming something at ISP). After the 12 minutes, it looks like PPPoE reestablished (IPCP: LayerUP, with the correct assigned IPv4 address), but apparently not fully, as I had "no interrnet". A few hours later, I can see my attempt to reload the interface, which also looks like it reestablished OK, then finally my reboot.
If someone wants to look at logs, I can share them somewhere private...
@dseven sure, send them to franco@opnsense.org but please also include the system log.
Thanks,
Franco
PS: One thing this could be is https://github.com/opnsense/core/pull/7692 which is being shipped in 24.7.2 next week.
That seems like it probably fits. I think it was actually the gateway, not the interface, that was red (did I mention I wasn't fully awake?). Emailed logs anyway. Thanks!
Yes, something about the default route here which goes through the gateway.
Was this a general issue or just IPv6? Because I can see IPv6 dropping out at some point not able to fix itself, but it seemed to recover IPv4. Or do I read this wrong?
Cheers,
Franco
Made a 24.7.1 backport of the commit to try: https://github.com/opnsense/core/commit/09209149
# opnsense-patch 09209149
Cheers,
Franco
It was a general issue. I didn't actually try IPv6, but v4 was down. Had I been awake, I would have checked if I had a default route for v4 - I'm guessing probably not.
I think I need to try to reproduce the issue - perhaps in my current patch state, then revert the patches from this thread (to rule them out as the cause), then try the backport. I'll hopefully get to work on that later today.
I found that reloading the PPPoE interface (from "Overview") resulted in the interface being up but no v4 default route (until I restarted the gateway monitor, then it came back). After applying the backported patch, that's no longer an issue - the default route gets reinstated after the PPPoE interface comes up, as expected. I'm fairly confident that this is the fix, and it doesn't seem to have anything to do with the patches for this thread. Apologies for the noise, and thanks!
Ok, nevertheless very nice to know that fix works :)
What is the CLI-Command to only remove a Patch?
Same as for applying. If the patch was applied, already, it will be reverted.
Yes, but make sure to revert them in the right order if you have multiple ones applied that build on each other (like this series here) otherwise the revert will fail for the right reason and leave you with without a code change.
Cheers,
Franco
Is there any hope of CLAT support for 25.1? Without it atm a true IPv6 deployment isn't possible without compatibility issues
It's in ipfw(4) but no significant questions about it.
I'm not sure how long-term smart it is to add NAT support in ipfw(4) alongside pf(4). But that's a mildly related story.
Cheers,
Franco
Hey Franco, I'm back from holidays and saw this. I upgraded to 24.7.1 but my kernel panic remains. Should I try this patch and could you confirm the patch for 24.7.1 please?
Thank you!!
Quote from: Hendre on August 18, 2024, 09:31:38 AM
Hey Franco, I'm back from holidays and saw this. I upgraded to 24.7.1 but my kernel panic remains. Should I try this patch and could you confirm the patch for 24.7.1 please?
The patches here have nothing to do with kernel, posting to a wrong thread perhaps?
Just asking to clarify. Thanks for letting me know it's not kernel related, will continue on the other thread!
My test:
- WAN: DHCPv6 - Prefix Delegation size: 64
- LAN: Track Interface - WAN
--> Result: this one happens frequently without any fix (I have read a lot about this one and try to do fix, it worked at previous version of OPNSense, but at this new version - nothing worked)
ISC DHCPv6/LAN: Available range No available address range for configured interface subnet size.
Please help.
Is there a new patch for 24.7.2 or is existing patch ok?
So far have reverted and applied 24.7.2
On a side note: I have seen one strange behaviour with the new IPv6 implementation (possible the new dhcp6c version):
With one of my ISPs (Deutsche Glasfaser), who uses DHCP and DHCPv6, I got no more IPv6 assigned until I enabled "Use IPv4 connectivity". That was not neccessary before. No big deal, however, just wanted to share that info.
But that's not a PPP(oE) deployment? The thing about IPv4 connectivity is that it is solely a PPP property but shown everywhere but practically useless.
Cheers,
Franco
That I wondered also about: As I wrote: DHCP/DHCPv6 only, but still, without the option, no IPv6. Started with 24.7.2.
It just needed a restart guess? :)
When I changed the interface setting, it immediately picked up the IPv6. Did that on two machines. On the third, I changed the setting before the 24.7.2 upgrade, that worked as well. But never "had" to set that checkbox before.
Unset and it keeps working?
https://github.com/opnsense/core/commit/805120ea7844f
The only thing that "dhcp6usev4iface" did was change the interface from the hardware to the PPP device when checked in get_real_interface(). When it's not PPP, it keeps (has to, really) the hardware name.
Cheers,
Franco
Maybe just a glitch or timing issue, but as I said, I had it on two machines with the same ISP. I know that this ISP sometimes takes a 2-3 minutes for DHCPv6, but the problem persisted for more than 5 minutes after the reboot.
Quote from: meyergru on August 22, 2024, 11:38:39 AM
Maybe just a glitch or timing issue, but as I said, I had it on two machines with the same ISP. I know that this ISP sometimes takes a 2-3 minutes for DHCPv6, but the problem persisted for more than 5 minutes after the reboot.
I had the exact same issue this morning also on a Deutsche Glasfaser circuit.
Opnsense autoupdate ran at 3am this morning. The problem is No IPv6 on WAN.
The Services dhcpd6, dpinger (v6) were stopped (crashed?) and can't be started anymore.
I found this thread and checking the "Use IPv4 connectivity" on the DHCPv6c fixed IP v6 instantly.
There is no PPPOE link.
I can provide any logs if needed.
> I found this thread and checking the "Use IPv4 connectivity" on the DHCPv6c fixed IP v6 instantly.
Ok, it *might* serialise the IPv6 to be after IPv4 but that's pretty annoying if the ISP enforces this.
Cheers,
Franco
Quote from: franco on August 22, 2024, 03:36:28 PM
> I found this thread and checking the "Use IPv4 connectivity" on the DHCPv6c fixed IP v6 instantly.
Ok, it *might* serialise the IPv6 to be after IPv4 but that's pretty annoying if the ISP enforces this.
Cheers,
Franco
Any logs you wish have? I can maybe setup a wireshark capture next week with the option set and also with option unset.
More Info: My Opnsense is running on Proxmox
The Update ran today at 3am and didn't pull an Ipv6 for 11 hours until 2pm as I set the WAN option to use Ipv4 and hit apply.
Hi,
Welcome back after short holidays 8)
As a reminder, Orange Poland ISP provides 2 PPPoE configs: IPv4-only and IPv6-only. Everything on top of VLAN (id 35)
IPv6-only provides link-local address and expects you to get IPv6 prefix via DHCPv6.
I did a clean install of 24.7 yesterday than upgrade to 24.7.2 and here are my observations:
- When PPPoE is configured for IPv4 I get IPv4 public address, dhcp6c.conf is configured properly (for PPPoE interface) but no prefix is recived - this is expected, no complain here.
- When PPPoE is configured for IPv6 and "Use IPv4 connection" is checked I get the v6 link-local address (form PPPoE) and no v4 address (as expected) but... no v6 prefix from DHCPv6 - dhcp6c.conf file is now empty.
- When PPPoE is configured for IPv6 and "Use IPv4 connection" is unchecked I get the v6 link-local address (form PPPoE) and no v4 address (as expected) no v6 prefix from DHCPv6 - dhcp6c.conf file configured properly except the interface is set now to VLAN interface as it should
For me it looks like without IPv4 address PPPoE is considered "down" and no dhcp6c config is generated?
> Any logs you wish have? I can maybe setup a wireshark capture next week with the option set and also with option unset.
In the scope of this PPP rework? Unfortunately not. The behaviour can only be observed. The logs will just tell you that the ISP doesn't answer if the DHCPv6 request is dispatched before DHCPv4 is complete.
> I did a clean install of 24.7 yesterday than upgrade to 24.7.2 and here are my observations:
Not surprising considering none of the work here is included in 24.7.2. ;)
Cheers,
Franco
Quote from: franco on August 22, 2024, 04:17:06 PM
> Not surprising considering none of the work here is included in 24.7.2. ;)
So what's the correct course of action in my case? Which patch should I apply to 24.7.2?
i am confused. "Deutsche Glasfaser" here as ISP as well, upgrading to 24.7.2 broke IPv6 DHCP completely. Tried using "Use IPv4 connectivity" which does not work in my case.
how do i fix this or revert to 24.7.1 quickly?
did you reboot?
Quote from: maclinuxfree on August 22, 2024, 07:22:59 PM
did you reboot?
yes, a few times already.
also not sure if relevant, but in my case there is no PPPoE. Both IPv4 and 6 are via DHCP. Whatever was changed in this version made something very odd.
for me it is working with dsl 250 and draytek modem..config is pppoev4 an dhcpv6
Quote from: CruxtheNinth on August 22, 2024, 07:17:35 PM
i am confused.
Apparently. This thread is about PPPoE WAN. You don't have any PPPoE set up, please start a new thread with whatever issues you are having.
Quote from: doktornotor on August 22, 2024, 07:31:18 PM
Quote from: CruxtheNinth on August 22, 2024, 07:17:35 PM
i am confused.
Apparently. This thread is about PPPoE WAN. You don't have any PPPoE set up, please start a new thread with whatever issues you are having.
yes, and i was using this thread because at least 2 other user with the same ISP has exactly the same issue after this upgrade.
Quote from: fatexs on August 22, 2024, 03:26:28 PM
Quote from: meyergru on August 22, 2024, 11:38:39 AM
Maybe just a glitch or timing issue, but as I said, I had it on two machines with the same ISP. I know that this ISP sometimes takes a 2-3 minutes for DHCPv6, but the problem persisted for more than 5 minutes after the reboot.
I had the exact same issue this morning also on a Deutsche Glasfaser circuit.
Opnsense autoupdate ran at 3am this morning. The problem is No IPv6 on WAN.
The Services dhcpd6, dpinger (v6) were stopped (crashed?) and can't be started anymore.
I found this thread and checking the "Use IPv4 connectivity" on the DHCPv6c fixed IP v6 instantly.
There is no PPPOE link.
I can provide any logs if needed.
Quote from: CruxtheNinth on August 22, 2024, 07:43:52 PM
yes, and i was using this thread because at least 2 other user with the same ISP has exactly the same issue after this upgrade.
The patches being tested here are for PPPoE connectivity. Additionally, they are NOT included in 24.7.2. This is not the thread to debug unrelated things.
Let's not make multiple wrongs a right. I want to continue testing PPP refactor patches.
To continue testing regression free changes on 24.7.2 community release please use:
https://github.com/opnsense/core/commit/77531748
https://github.com/opnsense/core/commit/7582088d
# opnsense-patch 77531748 7582088d
If you want to test full set of IPv6-only capability please use the development release bundled with 24.7.2 instead. There is actually a follow up patch that completely disables IPv4 in this case which isn't in the development release:
https://github.com/opnsense/core/commit/cb09c6671b
# opnsense-patch cb09c6671b
Cheers,
Franco
Hey Franco,
I upgraded to 24.7.2, updated my IPv6 to reproduce the kernel panic we've been talking about. I was able to reproduce it.
I then applied patch above, IPv6 restored and NO kernel panic.
Cheers,
Hendre
Quote from: franco on August 22, 2024, 08:11:33 PM
To continue testing regression free changes on 24.7.2 community release please use:
# opnsense-patch 77531748 7582088d
Still working just fine.
Should we ship both patches with 24.7.3 or do the first one in 24.7.3 and the second one in 24.7.4?
Cheers,
Franco
Well I'd say they make sense shipped together. Still wanted to test this with the VLAN tagging done on OPNsense instead of the bridged router if that's of any interest. Hopefully tomorrow, today's reboot quota exceeded 😂
Quote from: doktornotor on August 24, 2024, 09:37:41 PM
Well I'd say they make sense shipped together. Still wanted to test this with the VLAN tagging done on OPNsense instead of the bridged router if that's of any interest. Hopefully tomorrow, today's reboot quota exceeded 😂
Done for the sake of trying to reproduce this issue (https://forum.opnsense.org/index.php?topic=42120.0), no luck. It's working for me. (Will keep the setup, after firmware upgrades, the router defaults to bridge but no VLAN, so it's more convenient anyway).
I updated from the latest 24.1 to 24.7.2 (no patches from this thread applied). I'm cursed by an ISP supporting only 6rd. The 6rd tunnel configuration UI is completely missing from the PPPoE interface configuration in 24.7.2? The IPv6 option shows up as "None" after the update. The wan_stf tunnel device does remain configured through the upgrade process, and gets an address from the ISP, but isn't passing outbound v6 traffic from the LAN side, though v6 from the router itself gets out fine.
I didn't have a lot of time to diagnose and rolled back to 24.1 (thank you zfs boot environments!) when I have more time at the physical location of the router.
Will 6rd tunnel support over a PPPoE interface be returning in some form?
Quote from: seacycle on August 25, 2024, 08:38:24 PM
I updated from the latest 24.1 to 24.7.2 (no patches from this thread applied). I'm cursed by an ISP supporting only 6rd. The 6rd tunnel configuration UI is completely missing from the PPPoE interface configuration in 24.7.2?
I would suggest you comment on your use case on https://github.com/opnsense/core/issues/7446
Relevant commit: https://github.com/opnsense/core/commit/b2d9372b0f8b2583da90e1e294d88996539e1983
@seacycle
Thanks for reaching out. TLDR: you're looking for https://github.com/opnsense/core/commit/947e61b1a5
# opnsense-patch 947e61b1a5
The long version: Oh boy. This is one of the effects of confusion that "IPv4 connectivity" has created, because 6RD and 6TO4 do not work over PPPoE at all. So this is a side-by-side configuration, which is pretty mind-boggling considering your ISP goes through the effort to bring you online via PPPoE tunnel and gives you IPv6 outside the PPPoE tunnel... ok, why not? ;)
I'm going to assume 6RD still works on 24.7 despite the visibility glitch?
The important thing going forward is that these types of setups will no longer work on 25.1 in the way they are currently performed. In 25.1 you will have to delete 6RD from your WAN and create a separate "WAN6" interface from your "port" where PPPoE is running on (something like igb1 for example). There you chose IPv4 None and IPv6 6RD and it should work as before. That being said, the same should already work on all known OPNsense versions but it was favoured by be convenient two-in-one WAN configuration which, again, has been a source of great confusion for at least a decade.
Can you:
1. Confirm that the patch works? I'd add that to 24.7.3 of course.
2. Confirm that the configuration suggestions works as well on your end?
Please don't go, we need you for this. :)
Cheers,
Franco
Quote from: franco on August 26, 2024, 08:24:22 AM
which is pretty mind-boggling considering your ISP goes through the effort to bring you online via PPPoE tunnel and gives you IPv6 outside the PPPoE tunnel... ok, why not? ;)
Maybe they have done some "improvement" there - such as "no 6rd before you connect via PPPoE". ::) ;D
Quote from: doktornotor on August 26, 2024, 09:27:13 AM
Maybe they have done some "improvement" there - such as "no 6rd before you connect via PPPoE". ::) ;D
Please, don't even joke about it.
Cheers,
Franco
Quote from: franco on August 26, 2024, 09:46:19 AM
Please, don't even joke about it.
The IPv4 connectivity checkbox makes return... :-X ::)
As for the rest, cannot test 6rd with the suggested setup. Could do 6to4 but that seems to go to via he.net pretty much everywhere I did try with
traceroute 192.88.99.1, has been deprecated for 10 years or so and Google had hated it (https://support.google.com/websearch/thread/56943260/we-have-detected-you-are-using-a-6to4-tunnel-6to4-is-deprecated-and-will-be-shutdown-shortly?hl=en) years before they've started with the let's make HE.net tunnels unusable stunt (https://forums.he.net/index.php?topic=4253.0).
Quote from: franco on August 26, 2024, 08:24:22 AM
@seacycle
Thanks for reaching out. TLDR: you're looking for https://github.com/opnsense/core/commit/947e61b1a5
# opnsense-patch 947e61b1a5
The long version: Oh boy. This is one of the effects of confusion that "IPv4 connectivity" has created, because 6RD and 6TO4 do not work over PPPoE at all. So this is a side-by-side configuration, which is pretty mind-boggling considering your ISP goes through the effort to bring you online via PPPoE tunnel and gives you IPv6 outside the PPPoE tunnel... ok, why not? ;)
I've always felt the disconnect between the "simple" 6rd selection in the WAN interface configuration, and the actual underlying plumbing it sets up to be somewhat confusing. I do think the 6rd/6to4 configuration would make more sense relegated to the "Other Types" section along with GRE and friends, more or less completely independently configured, with its own assignment and firewall rules, as you describe, and would absolutely endorse that approach going forward.
6rd over pppoe it is the jankiest of all configurations an ISP could possibly offer for v6. But at about 2.7 million broadband subscribers CenturyLink isn't quite in the completely ignorable category here in the US. (And their symmetric fiber ipv4 performance, where I am, blows Comcast away, for less than half the price.)
Quote from: franco on August 26, 2024, 08:24:22 AM
I'm going to assume 6RD still works on 24.7 despite the visibility glitch?
I'm only occasionally at the physical location of this router, which makes testing different WAN configurations difficult, but I should be able to verify by Thursday. I've got 24.7.2 in loaded up in a separate boot environment so I can flip back and forth easily. What I recall from by brief testing was that (a) the wan_stf interface kept its configuration through the upgrade process and obtained a valid v6 prefix usable for outbound traffic from the router. But (b) that clients on the LAN side couldn't make v6 connections through the router to the outside. I didn't get as far as identifying where the failure was.
Quote from: franco on August 22, 2024, 08:11:33 PM
Let's not make multiple wrongs a right. I want to continue testing PPP refactor patches.
To continue testing regression free changes on 24.7.2 community release please use:
https://github.com/opnsense/core/commit/77531748
https://github.com/opnsense/core/commit/7582088d
# opnsense-patch 77531748 7582088d
If you want to test full set of IPv6-only capability please use the development release bundled with 24.7.2 instead. There is actually a follow up patch that completely disables IPv4 in this case which isn't in the development release:
https://github.com/opnsense/core/commit/cb09c6671b
# opnsense-patch cb09c6671b
Cheers,
Franco
Ok, I've applied the first two patches mentioned above and now I have no IPv6 connectivity. My ISP gives me a dynamic /56 prefix on WAN but with the patches I no longer get one. When trying to revert the patches I get 1 hunk failed for interfaces.inc
You need to revert them in reverse order I'd say.
Quote from: doktornotor on August 27, 2024, 07:18:27 PM
You need to revert them in reverse order I'd say.
That's exactly what I did and that's the end result.
Well, then don't do it reversed, ;D
Tried the old trick of pretending to change something to both LAN and WAN but saving with no changes, tried to apply and revert the patches again, now IPv6 works as usual. I have no idea why :))
LE: IPv6 connectivity dropped after a few minutes.
Is this on PPPoE or without? I can't tell exactly. There is something going on with dhcp6c as well, but all of this is hard to tell with 24.7.1/24.7.2 kernel being as flaky as it is. Tomorrow's 24.7.3 should make that much better.
In case it's dhcp6c try this:
# opnsense-revert -r 24.7.1 dhcp6c
(reboot needed)
Cheers,
Franco
PS: That being said I'm refraining from merging this due to IPv4 connectivity side effect for non-PPPoE. The first patch would make matters worse for these poor people relying on it.
Quote from: franco on August 26, 2024, 08:24:22 AM
@seacycle
Thanks for reaching out. TLDR: you're looking for https://github.com/opnsense/core/commit/947e61b1a5
# opnsense-patch 947e61b1a5
...
Can you:
1. Confirm that the patch works? I'd add that to 24.7.3 of course.
2. Confirm that the configuration suggestions works as well on your end?
Please don't go, we need you for this. :)
The patch does appear to work with the convoluted 6rd-over-pppoe-over-a-tagged-vlan that CenturyLink requires, thanks! (There are some issues with radvd not picking up the prefix reliably without a manual restart, but that pre-dates 24.7.)
Ok, thanks for confirming. This one will be in 24.7.3 today.
Interesting note about radvd. Are you using LAN interfaces to track the WAN? Just by memory I think it's true that for 6RD/6to4 there is no event to automatically reconfigure clients. Happy to fix that, but ideally with a ticket on GitHub.
I'd still like clarification on the second half:
Quote from: franco on August 26, 2024, 08:24:22 AM
The important thing going forward is that these types of setups will no longer work on 25.1 in the way they are currently performed. In 25.1 you will have to delete 6RD from your WAN and create a separate "WAN6" interface from your "port" where PPPoE is running on (something like igb1 for example). There you chose IPv4 None and IPv6 6RD and it should work as before. That being said, the same should already work on all known OPNsense versions but it was favoured by be convenient two-in-one WAN configuration which, again, has been a source of great confusion for at least a decade.
Can you:
[...]
2. Confirm that the configuration suggestions works as well on your end?
Thanks,
Franco
Quote from: franco on August 29, 2024, 07:32:38 AM
Interesting note about radvd. Are you using LAN interfaces to track the WAN? Just by memory I think it's true that for 6RD/6to4 there is no event to automatically reconfigure clients. Happy to fix that, but ideally with a ticket on GitHub.
Yes, tracking the WAN interface. I can put together a ticket.
Quote from: franco on August 29, 2024, 07:32:38 AM
I'd still like clarification on the second half:
Quote from: franco on August 26, 2024, 08:24:22 AM
The important thing going forward is that these types of setups will no longer work on 25.1 in the way they are currently performed. In 25.1 you will have to delete 6RD from your WAN and create a separate "WAN6" interface from your "port" where PPPoE is running on (something like igb1 for example). There you chose IPv4 None and IPv6 6RD and it should work as before. That being said, the same should already work on all known OPNsense versions but it was favoured by be convenient two-in-one WAN configuration which, again, has been a source of great confusion for at least a decade.
Can you:
[...]
2. Confirm that the configuration suggestions works as well on your end?
I agree that the two-in-one WAN configuration feels like an abstraction-too-far from the actual topology of the plumbing underneath, leading to confusion. I had thought that the stf tunnel setup would make more sense as an independent "Other Types" interface configuration; make an stf device, assign it to WAN6 and configure as 6rd there. Ultimately, it doesn't seem coupled to the pppoe interface by anything than by the routing table, as far as I understand.
I'm not 100% sure I follow how I would set up what you describe on 24.7, but would like to try it out. If I have:
igc1 - hardware interface connected to the provider's ONT (unassigned)
vlan01 - pppoe packates need to be tagged with vlan 201, so this is on igc1 (unassigned)
pppoe1 - on top of vlan01 (assigned as WAN)
igc1 and vlan01 are not currently assigned. Are you saying I'd drop the v6 configuration from WAN (pppoe1), assign vlan01 or igc1 to WAN6, and configure the 6rd from that?
> igc1 and vlan01 are not currently assigned. Are you saying I'd drop the v6 configuration from WAN (pppoe1), assign vlan01 or igc1 to WAN6, and configure the 6rd from that?
Correct. I believe it would be vlan01. This should confirm it for you from the console:
# ifconfig wan_stf
It seems a bit strange to split IPv4 and IPv6 interfaces but the reality is that it does this anyway even if it looks unified so the two interface approach should be fully compatible.
Cheers,
Franco
Quote from: franco on September 01, 2024, 08:34:09 AM
> igc1 and vlan01 are not currently assigned. Are you saying I'd drop the v6 configuration from WAN (pppoe1), assign vlan01 or igc1 to WAN6, and configure the 6rd from that?
Correct. I believe it would be vlan01. This should confirm it for you from the console:
# ifconfig wan_stf
Trial on 24.7.3_1:
- Removed the 6rd configuration from WAN (assigned to the pppoe interface), save, apply. The 6rd prefix is removed from the routing table, but the wan_stf interface remains partially configured (it has the tv4rbr address still configured, but no inet6 address) and is the tsill default v6 route.
- Reboot, flushes out the wan_stf interface and the associated default v6 route.
- Assign the vlan interface under the pppoe interface to WAN6, enable. IPv4 configured as none, IPv6 configured as 6rd, save apply. No stf interface gets configured. Also notice the old 6RD gateway associated with WAN is still listed, marked as defunct. Delete.
- Reboot, still no stf interface.
- Repeat, but use the hardware igc interface under the vlan interface (under the pppoe interface), same result.
- Remove the WAN6 assignment, re-configure 6RD on the WAN assignment (as it was originally), and the stf interface returns with a correct 6rd configuration.
Searching logs for WAN6 I see this a bunch of times:
/interfaces.php: The command '/sbin/ifconfig 'opt6_stf' inet6 description 'WAN6 (opt6)' up' returned exit code '1', the output was 'ifconfig: interface opt6_stf does not exist'So it seems like a code path to create the stf interface is getting missed in this scenario, but later code paths expecting to find it existing is being hit.
With 6rd configured on the WAN assignment (pppoe), the stf interface looks like:
wan_stf: flags=1004041<UP,RUNNING,LINK2,LOWER_UP> metric 0 mtu 1280
description: WAN (wan)
options=0
inet6 2602:<redacted>:: prefixlen 24
groups: stf
v4net 0.0.0.0/32 -> tv4br 205.171.2.64
nd6 options=103<PERFORMNUD,ACCEPT_RTADV,NO_DAD>
Thanks for testing. All of this is so silly it makes my head hurt.
You're probably seeing a log line that is this:
# opnsense-log | grep "not configuring 6RD tunnel"
So what this means is that 6RD is tied to an IPv4 mode and that should have been validated a long time ago. But it also means 6RD and 6TO4 go through the tunnel after all.
The good news is we don't have to worry about "IPv4 connectivity" knob in this case since we latch on by IP address and not device and so leave the 6RD/6to4 where it currently is. I'll add the required validation and that should be the end of it.
Thank you a lot for these insights,
Franco
PS: I was wrong and I'm glad we cleared that up. :)
Commit for reference only: https://github.com/opnsense/core/commit/c17d021b4c5
Ok here are the revised patches for 24.7.3:
https://github.com/opnsense/core/commit/6b28dae2
I've added dhclient compatibility for the "IPv4 connectivity" abuse situation by non-PPP setups. I've also fixed a stupid error WRT reading the actual setting correctly. This should address my current concerns and the reason why this fix didn't land in 24.7.3 so far. Although compatibility has been put back for 24.7 I still intend to remove support for non-PPP types in order to come up with an easier fix (because in this case we are not dealing with a volatile PPP interface),
https://github.com/opnsense/core/commit/82ea39b04c
Second commit remained as is.
# opnsense-revert opnsense && opnsense-patch 6b28dae2 82ea39b04c
Since 6b28dae2 is probably more likely to introduce issues it's my goal to bring it into 24.7.4. Some non-PPP testing is appreciated. 24.7.4 will also by default correctly deal and validate with 6RD/6to4 again.
82ea39b04c can follow in 24.7.5
Happy testing and thanks in advance.
Cheers,
Franco
Currently I have DHCP v6 being shut down due to strange behavior.
Just updated.
I see no difference in PPP behaviour since unpatched 24.7.3
Still have to do the vlan-jiggle to get PPP to come up, still have to manually start dhcp6, but overall a smooth update.
@HCB thanks, looking at your mail at this very moment. Sorry, it's been too busy.
I use a dynamic IPv4 connection and use 6TO4 for dynamic IPv6. Is it a good idea to test the current patchset, or are there any known issues for this kind of setup?
PPP-type IPv4? 6RD and 6to4 quirks have been ironed out due to previous feedback. I don't expect them to break.
At the moment the only patch to test is the second one, because the first one is in 24.7.4 now:
# opnsense-patch https://github.com/opnsense/core/commit/6592a9a69
At the moment there seems to be a problem with PPP gateway setting in the mpd.conf which needs a hotfix. But it didn't come up in the testing here, possibly because the thread is mostly about PPPoE.
Cheers,
Franco
I have some feedback regarding PPPoE + native IPv6 by DHCP. I noticed that when (I think) ISP forces PPPoE session refresh then OPNSense fails to renew IPv6 DHCP lease. I have static /56 and from my understanding of logs ppp log seems to be ok, but "opnsense", dhcp6c and rtsold, in consequence are not happy. Way around it is to refresh WAN interface configuration. This is not related to recent release. It happened even on 24.7 but it happens very early (I noticed too few occurrences to figure out a pattern)
Aside from that PPPoE rework works great on every release for me.
This is a general issue when the DHCPv6 server forgets the lease or rejects it. Does it not auto-recover after 10-30 minutes? I'm not suggesting that's ok, just trying to get a baseline since I've worked on fixes for dhcp6c regarding this situation.
Cheers,
Franco
Quote from: franco on September 13, 2024, 04:29:53 PM
This is a general issue when the DHCPv6 server forgets the lease or rejects it. Does it not auto-recover after 10-30 minutes? I'm not suggesting that's ok, just trying to get a baseline since I've worked on fixes for dhcp6c regarding this situation.
Cheers,
Franco
Haha, internet is so essential that I was not patient enough to try to see if this fixes itself in up to 30 minutes :D At least when it happened during the day. Come to think of it I don't know if it ever happened during the night and I haven't noticed. I will try to monitor that more closely.
EDIT: Actually I remembered one day waking up to internet being MIA and once I checked the logs and confirmed just now the session refresh >2h earlier and it didn't fix itself until WAN config re-save.
Thanks, we may be missing something in that case. Maybe the interface index shifts for PPPoE or the device is otherwise unavailable because it was recreated but still referencing the old one. For now I made a ticket:
https://github.com/opnsense/dhcp6c/issues/39
Since we had to hotfix 24.7.4 in this area the second patch backport doesn't apply. Here is an updated version:
# opnsense-patch https://github.com/opnsense/core/commit/d24300f9ab
Cheers,
Franco
Since the second patch is in 24.7.5 now I'll unsticky this thread now. Thanks all for the help!
Cheers,
Franco