Setting up fiber PPPoE connection

[markfree]

My current setup uses an ONU to bridge the ISPs fiber to my SOHO router, which handles the PPPoE connection.
Additionally, I had to input the GPON SN and the VLAN ID to connect the ONU.

I have a mini PC with an SFP+ port and I want to plug it directly into my ISP's fiber. However, I am not very familiar with OPNsense and I'm unsure if that is feasible.

Do you have any tips or tutorials to share regarding this?

[bartjsmit]

As long as the NIC in your mini PC is supported by FreeBSD 13.1 there is unlikely to be much bother with getting it to run as a WAN interface for OPNsense https://www.freebsd.org/releases/13.1R/hardware/#support

Just follow the regular installation instructions: https://docs.opnsense.org/manual/install.html

Bart...

[markfree]

My PC uses the "Mellanox ConnectX-3" controller which is listed as a supported network interface.

Code: [Select]

# pciconf -lv mlx4_core0
mlx4_core0@pci0:5:0:0:  class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0113
    vendor     = 'Mellanox Technologies'
    device     = 'MT27500 Family [ConnectX-3]'
    class      = network
    subclass   = ethernet
I did install OPNsense, but I'm having a hard time finding information about how to setup SFP with PPPoE.

So far, I have configured the SFP interface type as PPPoE.

After connecting the fiber to the interface, the logs show that the PPP interface is unable to connect.

Code: [Select]

<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="441"] [opt5_link0] Link: reconnection attempt 72
<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="442"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="443"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="444"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="445"] [opt5_link0] LCP: Down event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="446"] [opt5_link0] Link: reconnection attempt 73 in 3 seconds
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="447"] [opt5_link0] Link: reconnection attempt 73
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="448"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="449"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="450"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="451"] [opt5_link0] LCP: Down event
I am aware that my ISP's ONU uses a specific GPON SN and a VLAN id, but I don't know how to setup them on OPNsense.

[marcquark]

If you need a specific tagged VLAN you'll have to create a VLAN Interface on top of mlxen0 and then add your PPPoE interface to that

You can do a packet capture on mlxen0 to see how the frames leaving your interface look. Right now you'll probably see an untagged frame. After making the suggested changes you'll see it tagged with the VLAN ID you set for your VLAN interface

[markfree]

I have created a VLAN with id 600.


Then, I set up the VLAN as the WAN interface.


Finally, I configured the WAN interface to use PPPoE.


After applying that configuration, a PPPoE interface was created.


The WAN shows as UP on the Dashboard, but there's no Internet connection available.

[meyergru]

You got it wrong. The correct hierarchy would be:

WAN -> pppoe0 -> vlan01 -> mxlen0

where yours is:

WAN -> vlan01 -> mxlen0  and pppoe0 is on top of mxlen0, but isolated.

To fix this, all you have to do is to re-assign WAN to pppoe0 under "Interfaces -> Assignments".

[markfree]

I'm sorry, but I really don't get it.
As you can see from the previous pictures, VLAN is assigned to "mlxen0" as the parent interface.
And WAN interface is assinged to PPPoE. When I configured the "configuration type" to PPPoE, it automatically switched the assingment to PPPoE.

I've reset everything and tried again.

Here's what it looks like now:

Code: [Select]

LAN (igc0)      -> v4: 192.168.7.222/24
 OPT1 (igc1)     ->
 OPT2 (igc2)     ->
 SFP0 (mlxen0_vlan600) ->
 SFP1 (mlxen1)   ->
 WAN (pppoe0)    ->

[meyergru]

Then something does not add up in your images. vlan_wan.png shows WAN connected to vlan01, not pppoe0 as it should.

Under "Interface Assigments", the WAN entry's right side should read: "pppoe0 (vlan01) - WHATEVER", not "vlan01 (parent: mxlen0, Tag: 600)".

[markfree]

But, I thought picture "pppoe_wan.png" showed that PPPoE interface was created and assigned to WAN after configuring it.

Anyhow, this is the current assignment:


Also, the fiber is connected to the SFP0 interface, but it always shows the status of "no carrier". I'm not sure why.

Does that looks to be correct?

[meyergru]

Yes, the interface configuration does look correct like so.

But if the interface status is "no carrier", something seems to be wrong on the fiber side in the first place. Or maybe the SFP module is not recognized. I use a ZyXEL GPON SFP in my DEC 750 and when the fiber connection is online, I see this:



Some ISPs need the GPON serial, some also check more than that. It is always easier to rule out a misconfiguration when you eliminate factors first. My ISP provided a HUWAEI ONT with an ethernet port, so I tried that first. On those things, you can actually see the GPON status. With many SFP adapters, you have a web interface where you can see this as well.

In order to do this, you have to configure your mxlen0 to a static IP that matches the subnet of the SFP IP address. If you want to access the web interface, you have to create an outbound NAT rule, because there are no configurable routes on the SFP, so you need to "hide" behind your mxlen0 IP.

Then, you can access your SFP web interface. Mine shows this:



BTW: Lantiq-based ONT SFP modules like the ZyXEL only show an "up" status for the "ethernet" and allow web access if the fiber is attached and working. So there are three levels of connectivity: 1. fiber connection in order to be able to access the interface, 2. PLOAM status "up" for GPON connectivity (S/N and PLOAM password O.K.) and  3. Correct PPPoE connection via VLAN after authorization (some ISPs use DHCP at this level).

To even achieve a working GPON status was quite hard, as my ISP had a firmware on his ONUs that only worked for HUAWEI non-SFP ONTs - they actually checked the hardware and even the firmware version of the ONTs!

Only after they did some magic on their part was I able to connect with any ONT I wanted (the ISP-provided HUAWEI HG8012, another HUAWEI HG8010H, a ZyXEL PM3000-D20B SFP and a HUAWEI MA5671A SFP). This is possible because I can change the S/N on any of my own devices.

[markfree]

The ISP's fiber is definitely working and I'm connected through it.
When I connect the fiber to the SFP module, it does not show an up status. How can I make sure the module is detected?

Yes, my ISP requires a GPON serial number and I have it. In fact, my current ONU was configured with it.
So, are you saying that I should configure this serial number in the SFP module itself?

[meyergru]

Of course, if that is the only S/N that your ISP accepts. Either you copy the provider-supplied ONTs native S/N into your own equipment or you tell your ISP your S/N in order to unlock it (if they accept that - many do not). If you have done neither, this explains a lot, because in both cases, the GPON PLOAM status will never get "up" because the S/N does not match.

And as I wrote, even if the S/N matches, there may be more obstacles like the ISP not accepting anything but their own brand of ONTs or a need for a specific firmware version.

The process of setting the PLOAM password (if needed) and the S/N differs a lot between different brands of SFP ONTs. Some have a web GUI, some have telnet access. Some need a combination, e.g. my ZyXEL shows both on the web GUI:



But only the PLOAM password (called "SLID" here) is changeable in the GUI. The procedure to set the S/N is via telnet, see: https://github.com/xvzf/zyxel-gpon-sfp. You can see I actually chose a S/N that would normally be a HUAWEI one ("HWTC...."). That is why I can use the original ONT as a backup - both devices have the same S/N (most ISPs will only register one S/N).

[markfree]

I see what you mean. My current ONU is a TP-Link XZ000-G3 and I'm able to input the GPON SN and password. I haven't had any issues with it so far.

The PC SFP module is a Miljet MJ-SFPGE-BXD-20S and I could not find any information about its web interface or any configuration of it.
It seems that the module is not a GPON SFP after all. So, there is no GPON SN to configure on it.

At least, I found this interesting source of ONT "hacks", https://hack-gpon.github.io/.

Now, I'll look for the correct module.
Thank you for your insights.

[meyergru]

You are mixing incompatible technologies: The Miljet module just has the same single mode fiber as a physical transport as the TP-Link, even the TX wavelengths differ (i.e. the LED light colour is different). What is far worse is that the Miljet is not a GPON SFP module.

Somehow this is like connection a water hose to an gas outlet: It may physically fit and both transport "fluids", but despite that, it will not work. Or to illustrate it:



You have realized by now that this is not an OpnSense problem. We are now at least two layers apart in terms of the OSI model. The hack gpon site you found is a good resource to find info about suitable modules whose S/N can be changed easily. These also differ in the plugs used. SC is the most widely deployed type and it is used in your ZyXEL, but I could not find if it is SC/APC (green plug) or SC/UPC (blue plug) (see https://support.zyxel.eu/hc/de/articles/360005173700-Beschreibung-der-in-GPON-Ger%C3%A4ten-verwendeten-optischen-Anschl%C3%BCsse-SC-APC-und-SC-UPC for the difference). You should look for a GPON SFP module that is compatible with your optical cabling - often, they are available in both flavours.

But remember: Having a real GPON module is a necessary but not necessarily sufficient precondition for this to work - alas your ISP may have other hurdles in stock (mine did!).

[markfree]

I've managed to acquire a GBIC GPON OLT that should have a Web interface with the address 192.168.1.1, but I couldn't connect to the module yet. The module is the ODI Realtek DFP-34X-2C2.
I'm not sure how to make the necessary outbound NAT rule.

The assignments are as follows.
 LAN (igc1)      -> v4: 192.168.1.6/24
 OPT2 (mlxen0)   -> v4: 192.168.1.1/24
 PC -> 192.168.1.3/24
 
For now, I've set the outbound NAT as Hybrid and created the following rule, but it doesn't seem to work.


Any tips?