PPPoE, VLAN & RFC4638
Most modern Internet Broadband connections (VDSL, Fiber) are based on ethernet instead of ATM, but still using PPP for connectivity and subscriber management. Many ISP's also offering extra services (besides IPv4 & IPv6 Internet) like IPTV and VOIP, most of the time provided via different VLAN's.
A PPP header is 8 bytes, so when you're using PPP over Ethernet your (usable) MTU size will be slightly lowered from it's default (1500) to 1492. I'm not going to explain here why this might be problematic or at least suboptimal in a standard Ethernet network, DuckDuckGo is your friend ;-). Try: "MTU and Fragmentation" if you like to learn more about this topic.
Now the good news is that configuring PPPoE with VLAN's is extremly simple using OPNSense. Even better, if your ISP supports RFC4638 (Baby Jumbo's) on it's PPPoE connection, things get really awesome.....
"To Whom It May Concern":
The instructions below are for OPNSense configurations that are DIRECTLY connected to an ISP infrastrucure and terminate the PPPoE connection at the OPNSense box itself or using a CPE (modem) that provides PPPoE passthrough functionality.
MAC Spoofing:
If you need to use a custom MAC Address, configure it on the physical interface (01:01:01:01:01:01 in this example). When you don't know if you need a custom MAC, you probably don't need one, just leave this item blank.
MTU Size:
If your ISP supports RFC4638, configure the MTU size on the Physical NIC (1512) and the PPPoE VLAN (1508), otherwise leave these two settings blank. If your ISP supports RFC4638, but it doesn't work, first configure everything WITHOUT custom MTU settings, this should work anyway. Double check if your physical NIC supports custom MTU (>1500) sizes.
1 - Configure your WAN interface
- Unnumbered
- Custom MAC Address (if needed)
- Custom MTU Size (if supported by NIC & ISP)
Login to view screenshot:
2 - Add Service VLAN (IPTV)
- Use WAN Interface as parent
- My ISP is using VLAN4 for IPTV
- Check YOUR ISP instructions for the right VLAN id
Login to view screenshot:
3 - Add Service VLAN (IP)
- Use WAN Interface as parent
- My ISP is using VLAN6 for regular IPv4 & IPv6 Internet
- Check YOUR ISP instructions for the right VLAN id
Login to view screenshot:
4 - Both Service VLAN's configured
Login to view screenshot:
5 - Assign the configured Service VLAN Interfaces
Login to view screenshot:
6 - Configure the VLAN (IP)
- Configure IPv4 Configuration Type: PPPoE
- Custom MTU Size (if supported by NIC & ISP)
Login to view screenshot:
7 - Configure the VLAN (IP) - PPPoE
- Provide username & password for the PPPoE configuration
- A Service Name is not required in most cases
Login to view screenshot:
8 - OPNSense Automagicly configured your PPP Interface
Login to view screenshot:
9 - Check, Check, Double Check
Login to view screenshot:
10 - Configure the VLAN (IPTV)
- If needed configure the other Service VLAN
- My ISP is using DHCP for the IPTV Service VLAN, with some custom request settings
- Check YOUR ISP instructions for the required DHCP options
- I don't use IPTV, so no futher details
Login to view screenshot:
Really, that's all...
Check your PPPoE Connection from "Interfaces -> Overview -> PPPOE-INTERFACE"
If everything went well, you see a MTU size of 1500 and an active IPv4 address.
Validated with OPNsense 21.1 - It works for Me!!!
Thanks for the guide, very useful. Quck one: why don't you set the physical WAN at 1508 too? Why 1512?
Thanks.
Quote from: hushcoden on January 31, 2021, 02:51:01 PM
Quick one: why don't you set the physical WAN at 1508 too? Why 1512?
Extra 4 bytes for the 802.1q VLAN
Great HowTo
-fd0
So if I understand this correctly and I already have my PPPoE set up and that hopefully my ISP supports this, all I need to do is adjust the MTUs at the same places? I don't have one for VoIP so I can just adjust things for the two interfaces.
Thanks! This post made me realize I needed to enable the phisical interface in order to change the MTU to 1508 (in my case). I thought I already implemented rfc4638 by changing the settings on the WAN interface (vlan). Thanks again!
Sorry for bring this up after so many time, but this config works if the ISP also provides IPv6?
thanks for this detailed guide. Does this stil work on OPNsense 23.7.3-amd64
I have configured it like you did and WAN_FTTH MTU 1508, but at interfaces-overview I still see: MTU 1492 ?
A reboot did not help either.
It's being worked on at the moment. I assume some sort of driver prevents to change the MTU... but it's not easily reproducible. Legend has it the development version already works, but only one person did confirm this briefly.
Cheers,
Franco
Thanks Franco for the fast reply and good to know, while waiting the next update :-).
Depending on the lack of feedback on the development version the stable rollout will likely be paused as well.
So we talk about some time in October perhaps.
Cheers,
Franco
is there a patch I can run to check on OPNsense 23.7.3-amd64 if it works?
Sure, here is the backport for 23.7.3:
https://github.com/opnsense/core/commit/f934046cb47
# opnsense-patch f934046cb47
Cheers,
Franco
applied the patch
opnsense-patch f934046cb47
Fetched f934046cb47 via https://github.com/opnsense/core
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From f934046cb478ceb8116721039c3d0ed17f62c5d2 Mon Sep 17 00:00:00 2001
|From: Franco Fichtner <franco@opnsense.org>
|Date: Fri, 8 Sep 2023 13:08:45 +0200
|Subject: [PATCH] interfaces: implement interface_configure_mtu()
|
|---
| src/etc/inc/interfaces.inc | 54 +++++++++++++++++++++-----------------
| 1 file changed, 30 insertions(+), 24 deletions(-)
|
|diff --git a/src/etc/inc/interfaces.inc b/src/etc/inc/interfaces.inc
|index 49c2d45fc3..7b4f637c54 100644
|--- a/src/etc/inc/interfaces.inc
|+++ b/src/etc/inc/interfaces.inc
--------------------------
Patching file etc/inc/interfaces.inc using Plan A...
Hunk #1 succeeded at 2172.
Hunk #2 succeeded at 2213.
Hunk #3 succeeded at 2221.
Hunk #4 succeeded at 2304.
Hunk #5 succeeded at 2373.
done
All patches have been applied successfully. Have a nice day.
Did a reboot, and mtu is showing correct now at interfaces-overview MTU 1500! THNX!
Quote from: franco on September 08, 2023, 01:15:08 PM
Sure, here is the backport for 23.7.3:
https://github.com/opnsense/core/commit/f934046cb47
# opnsense-patch f934046cb47
Nice!
Applied the patch on 23.7.3 with pppoe on top of igb interface (Intel 350), works like a charm...
hello! Sorry for bringing up this topic again, but I've tried to set my PPPoE at 1500, but looks like no matter how I set my phisical interface + vlan, the pppoe is always created using 1492.
I'm using i225 nics and OPNsense 23.7.4-amd64, isp supports (mtu) 1500 bytes (rfc4638)
May I ask a little of help on how to applied the patch mentioned before?
https://github.com/opnsense/core/commit/f934046cb47
Thanks in advance!!
F
edit:
I sshed, and just push "opnsense-patch f934046cb47" and it worked :) Now pppoe is 1500. Thanks Franco!
Ok wasn't fast enough. 8)
This will be included in 23.7.5 tomorrow.
Cheers,
Franco
Quote from: franco on September 25, 2023, 07:58:08 PM
Ok wasn't fast enough. 8)
This will be included in 23.7.5 tomorrow.
Cheers,
Franco
Excellent, thanks.
Question - if we have previously enabled the parent igb interface just to set MTU to 1508 (and thus creating an annoying superfluous interface listing throughout firewalls etc.) can we now remove said interface in 23.7.5? If so, safest way of doing so?
Thanks, as ever, for your tireless efforts!
The code will try to calculate the requirement for the parent and set it if it needs to be increased. If the parent is assigned it makes sure to look at the custom MTU value there and use that (even if it's too small).
The best course of action is to remove the parent assignment and let the function take are of it. This has the benefit of adapting to new situations automatically. If the parent can't be deleted for other reasons clearing the MTU value is the equivalent of deleting the parent from the scope of the MTU function.
Cheers,
Franco
QQ: My WAN interface has a 'parent' interface with an IP address on the same subnet of my modem, and I've created that interface to connect to the web gui of the modem.
I've now changed the WAN port MTU to 1508 -> the calculated MTU goes from 1492 to 1500 and the MTU of the parent interface shows 1508. So far so good, BUT I noticed in the interface statics that the parent interface has a never stop number of 'errors in' and 'collision': should I worry about or not?
Tia.
Quote from: hushcoden on October 13, 2023, 06:56:08 PM
I've now changed the WAN port MTU to 1508 -> the calculated MTU goes from 1492 to 1500 and the MTU of the parent interface shows 1508. So far so good, BUT I noticed in the interface statics that the parent interface has a never stop number of 'errors in' and 'collision': should I worry about or not?
Yeah, you should.... But I would be surprised if this is related to this MTU change if your ISP is supporting RFC4638 and you're sure (double check) your NIC _supports_ >1500 MTU sizes. You can simply check by reverting the MTU change and see if these errors are still there in default (1492) mode.
Collision errors are in most cases related to duplex errors, check if your NIC speed & duplex settings are aligned with switch port or modem port. Auto Negotiation shouldn't be a problem for decades, but funky NIC's (Realtek) and cheap "smart" switches can still be trouble makers. You can try to set the speed & duplex settings fixed on _BOTH_ sides manually (10/100/1000Mb / Half/Full-Duplex), just be sure _BOTH_ sides are configured exactly the same (Fixed or Auto).
If you fix the collisions you probably fix the errors with it, but one step at a time ;).
Thanks netnut but indeed I started to see such errors just after I entered the MTU value 1508 on the WAN port - and just to be sure I was clear, those errors and collisions are on the parent interface and not on the WAN interface itself which luckily still shows no errors/collisions.
The NICs are I225-V and I'm pretty sure they support jumbo frames.
So, are you saying that this is a 'serious' issue even if it's on the parent interface of WAN?
P.S.
I tried to use VIP to accomplish the same result, but no luck at all, unfortunately, so this is for me the only way I can access the modem web GUI.
Quote from: hushcoden on October 15, 2023, 12:45:45 PM
The NICs are I225-V and I'm pretty sure they support jumbo frames.
These go up to 9.5KB Jumbo's, so yeah, that NIC shouldn't be a problem. Are you really sure your ISP supports RFC4638 ? It's common, but certainly not default.
Quote
So, are you saying that this is a 'serious' issue even if it's on the parent interface of WAN?
Well, don't panic ;). But errors are BAD, no errors are GOOD :D.
Since the reboot for the v23.7.5 upgrade 19 days ago (and a non-reboot upgrade to v23.7.6) my pppoe and parents showing zero collisions/errors, that should be the norm...
Having collisions means something is sub optimal (or Realtek ;)), normally a collision gets resend so the effect is not always noticeable, especially on a single interface ISP uplink.
What does "Media" in "INTERFACES: OVERVIEW" shows you at the physical parent interface ?
Other common things to check are the quality of your patch cable (try a non-self-made > CAT5e), and the connector (pull in/out and check if it fits firmly on both sides).
Quote from: netnut on October 15, 2023, 11:14:51 PM
What does "Media" in "INTERFACES: OVERVIEW" shows you at the physical parent interface ?
1000baseT <full-duplex>
Quote
Other common things to check are the quality of your patch cable (try a non-self-made > CAT5e), and the connector (pull in/out and check if it fits firmly on both sides).
Tested and it's all good.
Anyway, I changed the WAN interface MTU value back to 1492 and 'magically' no more errors in/collisions for the parent interface ::)
Also, in order for the parent interface to get back to MTU=1500 I had to restart OPNsense, but I recall in the previous versions that was happening 'on the fly'.
Quote
Anyway, I changed the WAN interface MTU value back to 1492 and 'magically' no more errors in/collisions for the parent interface
You still didn't confirm if your ISP is supporting RC4638, which is mandatory to make "this" work.
The only last suggestion I have, when your are using PPPOE on top of a VLAN, try to set the MTU on the physical parent interface to 1512. If that doesn't work I'm out of options... :)
I did contact my ISP to confirm that, but no response so far. Besides, I had to replace my modem and the new one doesn't support MTU higher than 1500, so never mind I can live with 1492 ;D
Many thanks !
Does this patch apply to '23.7.10_1-amd64' - I seem to be getting something like 4 out of 5 failed block messages.
Quote from: truesword88 on January 03, 2024, 04:30:25 AM
Does this patch apply to '23.7.10_1-amd64' -
https://forum.opnsense.org/index.php?topic=21207.msg176018#msg176018
Quote
I seem to be getting something like 4 out of 5 failed block messages.
Block messages ? This is about MTU, not packet filter.
The exact message when trying to apply the patch is: 'hunks' - 4 out of 5 hunks failed, see attached image.
It was at this point, it occurred to me that the patch might only apply to 23.7.3 & not 23.7.10_1..
Patches eventually reach the stable release. I think that most of the time patching something to make it work may alter the success rate in the future considerably.
To my knowledge there are no more complaints about PPPoE MTU since we worked on it in early 23.7.x.
Cheers,
Franco
Newbie here. If my isps config uses only vlan tag do I follow iptv or vlan ipv4/6 config ? And BTW I found this info at the router config cause I got no manual from my isp or any kind of instructions neither for oknsense neither for what to set at each label.
Quote from: Marinoz on January 21, 2024, 07:08:47 PM
Newbie here. If my isps config uses only vlan tag do I follow iptv or vlan ipv4/6 config ? And BTW I found this info at the router config cause I got no manual from my isp or any kind of instructions neither for oknsense neither for what to set at each label.
If your ISP is using VLAN's you need to know which VLAN ID's for which service upfront. This How-To is using VLAN ID 4 & 6 for IP connectivity and IPTV, but this will/may be different for each and every other ISP.
The only thing that matters is the VLAN ID of the VLAN interface you create, it should match the ID your ISP is using for the service you like to consume.
You're free to name this interface anything you like. The required IP configuration on this interface also depends on the specs and requirements of your ISP (ie. Static, DHCP, PD etc)
Quote from: netnut on January 23, 2024, 10:40:33 PM
Quote from: Marinoz on January 21, 2024, 07:08:47 PM
Newbie here. If my isps config uses only vlan tag do I follow iptv or vlan ipv4/6 config ? And BTW I found this info at the router config cause I got no manual from my isp or any kind of instructions neither for oknsense neither for what to set at each label.
If your ISP is using VLAN's you need to know which VLAN ID's for which service upfront. This How-To is using VLAN ID 4 & 6 for IP connectivity and IPTV, but this will/may be different for each and every other ISP.
The only thing that matters is the VLAN ID of the VLAN interface you create, it should match the ID your ISP is using for the service you like to consume.
You're free to name this interface anything you like. The required IP configuration on this interface also depends on the specs and requirements of your ISP (ie. Static, DHCP, PD etc)
hello its marinoz but i cant access my other account at the time. How can i know what is the vlan used for? i just ask to name it correctly although there is no need
Quote from: Marinas on January 31, 2024, 12:10:59 PM
hello its marinoz but i cant access my other account at the time. How can i know what is the vlan used for? i just ask to name it correctly although there is no need
The only existing way is to ask your ISP which VLAN they are using and if at all.
@netnut Any reason that you have the WAN interface disabled? Also can you please elaborate your assignments on picture 5? Currently my assignments are these https://i.ibb.co/fHF7cVY/Assignments.png.
Quote from: panseit on March 01, 2024, 03:48:33 PM
@netnut Any reason that you have the WAN interface disabled?
The WAN interface isn't disabled (as seen at screenshot 1), I just renamed my physical WAN interfaces to WAN_1 and WAN_2 because I use multiple WAN uplinks. Be aware that interface naming is arbitrary with OPNsense, so you could name them anyway you want, of course it makes sense to select reasonable names.
In this example I used the physical WAN_1 interface for the PPPoE config with the two vlans.
Quote
Also can you please elaborate your assignments on picture 5? Currently my assignments are these https://i.ibb.co/fHF7cVY/Assignments.png.
The screenshots slightly differs from yours because these were made with an older version of OPNsense. I believe somewhere in 22.07 OPNsense switched from a one-to-one VLAN Interface <-> VLAN ID naming to a much more flexible naming where the VLAN interfaces are named incremental and don't have a fixed relation with the VLAN ID.
Will update the screenshots to avoid confusion, thanks for the reminder 8)
Quote from: netnut on March 02, 2024, 12:13:34 AM
Quote from: panseit on March 01, 2024, 03:48:33 PM
@netnut Any reason that you have the WAN interface disabled?
The WAN interface isn't disabled (as seen at screenshot 1), I just renamed my physical WAN interfaces to WAN_1 and WAN_2 because I use multiple WAN uplinks. Be aware that interface naming is arbitrary with OPNsense, so you could name them anyway you want, of course it makes sense to select reasonable names.
In this example I used the physical WAN_1 interface for the PPPoE config with the two vlans.
Quote
Also can you please elaborate your assignments on picture 5? Currently my assignments are these https://i.ibb.co/fHF7cVY/Assignments.png.
The screenshots slightly differs from yours because these were made with an older version of OPNsense. I believe somewhere in 22.07 OPNsense switched from a one-to-one VLAN Interface <-> VLAN ID naming to a much more flexible naming where the VLAN interfaces are named incremental and don't have a fixed relation with the VLAN ID.
Will update the screenshots to avoid confusion, thanks for the reminder 8)
I meant the Ipv4 and IPv6 Configuration on WAN_1. Then you just configure wan_ftth. Couldn't you just configure wan_1?
Where do I configured unnumbered IP from ISP? My ISP provide me with /29 static IP address.
Thx for the guide.
I have a question my PPPoE seems to connect most of the times. They is times where it just doesnt wanna connect. If i reboot a couple times it works. What can that be?
Quote from: netnut on January 31, 2021, 01:09:24 AM
5 - Assign the configured Service VLAN Interfaces
Login to view screenshot:
From a security perspective: Is it important to leave one Interface assignment (in your case WAN_1) on the physical and untagged network interface (in your case igb4). Maybe to block untagged traffic or so?!
Or could you also reassign your [WAN] Interface to the pppoe device?
Quote from: Mindflayer on December 04, 2024, 10:32:11 PM
From a security perspective: Is it important to leave one Interface assignment (in your case WAN_1) on the physical and untagged network interface (in your case igb4). Maybe to block untagged traffic or so?!
Or could you also reassign your [WAN] Interface to the pppoe device?
OPNsense will silently drop any traffic arriving on an unassigned interface.
Not using/assigning untagged together with tagged VLANs on the same physical IF has been the general recommendation as long as I remember.
A trunk port is a trunk port and an access port is an access port. On switches that demand *some* "native VLAN" or "PVID" whatever they call it, I usually assign something like 99 or 999 which is not used to carry traffic anywhere in my network.
But even Mikrotik has got an "only tagged" setting for ports.
Quote from: Mindflayer on December 04, 2024, 10:32:11 PM
From a security perspective: Is it important to leave one Interface assignment (in your case WAN_1) on the physical and untagged network interface (in your case igb4). Maybe to block untagged traffic or so?!
Or could you also reassign your [WAN] Interface to the pppoe device?
I'm not sure if I understand your question completley, but a pppoe device is assigned TO an interface. Because this HowTo is about PPPoE in combination with a provider based service VLAN, the pppoe device is assigned to the VLAN interface derived from the physical WAN interface. It's basically like a layered cake:
You start with a physical WAN interface, in this HowTo that's "igb4". Because multiple WAN interfaces are used (two different providers) this physical interface is renamed to WAN_1 (the other one WAN_2, but out of scope in this example). This interface is specifically enabled in OPNsense to assign custom MAC and MTU values, but at least the MTU can be calculated in other ways with recent OPNsense versions. (instructions are made on older version).
Because you need to match the VLAN of your provider you now need to create a (tagged) VLAN interface on top of the igb4 / WAN_1 interface AND name it too, in this example the specific VLAN interface is named WAN_1_FTTH for generic ISP IP services.
The actual PPPoE interface (pppoe0) is configured on top of this new VLAN interface (WAN_1_FTTH) and named/assigned automatically by OPNsense. In the example there's also a second VLAN interface on the same physical WAN interface for IPTV services, this doesn't use PPPoE but raw Ethernet/IP.
So the full example ends up with three interfaces that are ISP oriented, the raw ethernet WAN interface and two VLAN interfaces, one using PPPoE for generic IP services (ie. Internet) and one standard VLAN IP interface. Yes, the raw ethernet WAN interface can be seen as an untagged interface towards your ISP, but it's unnumbered (doesn't have an active IP address assigned) so can't be used in > Layer 3 communication. I can't think of a Layer 2 attack surface that compromises the security of the WAN interface in this scenario, except for some L2 DOS stuff, but that would compromise the core service of my ISP, they probably detect that earlier than I can/do.
So your layered PPPoE VLAN Cake looks like this:
Raw Ethernet Interface -> VLAN Interface -> PPPoE Interface -> IP
@netnut this last post is incredibly good to understand the architectural principles of the setup.
May I suggest you add them to the original post? It will only enrich it, greatly.
p.s. my connections are FTTP aka FTTH and luckily without PPP nor VLANs. I still find it useful as reference.
Quote from: netnut on December 05, 2024, 05:02:30 PM
I'm not sure if I understand your question completley, but a pppoe device is assigned TO an interface. Because this HowTo is about PPPoE in combination with a provider based service VLAN, the pppoe device is assigned to the VLAN interface derived from the physical WAN interface.
...
@netnut: First of all: Thank you for your fast answer!
I think you really did not understand my question completely, but may be my fault.
What I means is:
On this picture: https://forum.opnsense.org/index.php?action=dlattach;topic=21207.0;attach=14911;image
I can see that you created a VLAN named "WAN_1_FTTH" AND you assigned that VLAN to a new interface (unfortunately) also named "WAN_1_FTTH". I tried around for a very long time with the Interfaces in OPNsense, and I think that this second step of creating an Interface for the VLAN is not necessary when you want to use PPPoE.
The layered cake is a good analogy.
- You have a physical adapter called "igb4". It is not necessary to assign that physical adapter to an interface.
- Then you create a VLAN ontop of that physical adapter. It is again not necessary to assign that VLAN to an Interface.
- Then you create a Point-To-Point device (configured as PPPoE) ontop of the VLAN. Only this Point-To-Point device, you have to assign to a (new) Interface.
- OPNsense then automatically detects/creates a Gateway, which is based on that Interface, which is based on the Point-To-Point device.
And now my question is: Does it have any downsides, from a security perspective, if I dont assign/create an Interface for the igb4 (step 1) AND the VLAN (step 2)?
No it does not and you need not and IMHO should not assign these interfaces.
Quote from: Patrick M. Hausen on December 05, 2024, 06:47:28 PM
No it does not and you need not and IMHO should not assign these interfaces.
What's the reason that you think it
should not be done? Only because you have 1 (or 2) more (unused) Interfaces?
Quote from: Mindflayer on December 05, 2024, 06:46:21 PM
And now my question is: Does it have any downsides, from a security perspective, if I dont assign/create an Interface for the igb4 (step 1) AND the VLAN (step 2)?
No. The only reason you would assign the interface is if you want to change any property, like the MAC address or MTU.
Although there's already an option in OPNsense to calculate / compensate for the MTU size with PPP connections, it assumes PPPoE over a raw ethernet interface (1500 (ethernet) + 8 (ppp) = 1508). Where I like to set the raw parent at 1512 (1500 (ethernet) + 8 (ppp) + 4 (vlan) = 1512), for this I need to enable the parent to set these values. I didn't look into recent OPNsense versions regarding automagic PPP MTU settings (I beiieve introduced in 24.07), still using my original install from years ago (and upgraded it with every release instead of fresh install).
@cookiemonster
A slightly updated version with screenshots of a more recent OPNsense version (with the changed internal VLAN assignment logic) is on my todo list. Will be busy for some more weeks, maybe a nice task when I try to avoid my mother-in-law during christmas dinner... ;-)
Quote from: Mindflayer on December 05, 2024, 06:48:23 PM
Quote from: Patrick M. Hausen on December 05, 2024, 06:47:28 PM
No it does not and you need not and IMHO should not assign these interfaces.
What's the reason that you think it should not be done? Only because you have 1 (or 2) more (unused) Interfaces?
That and a possible way for accidental misconfiguration that might weaken your setup. What isn't assigned will not carry traffic.