My current setup uses an ONU to bridge the ISPs fiber to my SOHO router, which handles the PPPoE connection.
Additionally, I had to input the GPON SN and the VLAN ID to connect the ONU.
I have a mini PC with an SFP+ port and I want to plug it directly into my ISP's fiber. However, I am not very familiar with OPNsense and I'm unsure if that is feasible.
Do you have any tips or tutorials to share regarding this?
As long as the NIC in your mini PC is supported by FreeBSD 13.1 there is unlikely to be much bother with getting it to run as a WAN interface for OPNsense https://www.freebsd.org/releases/13.1R/hardware/#support
Just follow the regular installation instructions: https://docs.opnsense.org/manual/install.html
Bart...
My PC uses the "Mellanox ConnectX-3" controller which is listed as a supported network interface.
# pciconf -lv mlx4_core0
mlx4_core0@pci0:5:0:0: class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0113
vendor = 'Mellanox Technologies'
device = 'MT27500 Family [ConnectX-3]'
class = network
subclass = ethernet
I did install OPNsense, but I'm having a hard time finding information about how to setup SFP with PPPoE.
So far, I have configured the SFP interface type as PPPoE.
After connecting the fiber to the interface, the logs show that the PPP interface is unable to connect.
<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="441"] [opt5_link0] Link: reconnection attempt 72
<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="442"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="443"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="444"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="445"] [opt5_link0] LCP: Down event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="446"] [opt5_link0] Link: reconnection attempt 73 in 3 seconds
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="447"] [opt5_link0] Link: reconnection attempt 73
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="448"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="449"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="450"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="451"] [opt5_link0] LCP: Down event
I am aware that my ISP's ONU uses a specific GPON SN and a VLAN id, but I don't know how to setup them on OPNsense.
If you need a specific tagged VLAN you'll have to create a VLAN Interface on top of mlxen0 and then add your PPPoE interface to that
You can do a packet capture on mlxen0 to see how the frames leaving your interface look. Right now you'll probably see an untagged frame. After making the suggested changes you'll see it tagged with the VLAN ID you set for your VLAN interface
I have created a VLAN with id 600.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=27677)
Then, I set up the VLAN as the WAN interface.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=27681)
Finally, I configured the WAN interface to use PPPoE.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=27683)
After applying that configuration, a PPPoE interface was created.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=27679)
The WAN shows as UP on the Dashboard, but there's no Internet connection available.
You got it wrong. The correct hierarchy would be:
WAN -> pppoe0 -> vlan01 -> mxlen0
where yours is:
WAN -> vlan01 -> mxlen0 and pppoe0 is on top of mxlen0, but isolated.
To fix this, all you have to do is to re-assign WAN to pppoe0 under "Interfaces -> Assignments".
I'm sorry, but I really don't get it.
As you can see from the previous pictures, VLAN is assigned to "mlxen0" as the parent interface.
And WAN interface is assinged to PPPoE. When I configured the "configuration type" to PPPoE, it automatically switched the assingment to PPPoE.
I've reset everything and tried again.
Here's what it looks like now:
LAN (igc0) -> v4: 192.168.7.222/24
OPT1 (igc1) ->
OPT2 (igc2) ->
SFP0 (mlxen0_vlan600) ->
SFP1 (mlxen1) ->
WAN (pppoe0) ->
Then something does not add up in your images. vlan_wan.png shows WAN connected to vlan01, not pppoe0 as it should.
Under "Interface Assigments", the WAN entry's right side should read: "pppoe0 (vlan01) - WHATEVER", not "vlan01 (parent: mxlen0, Tag: 600)".
But, I thought picture "pppoe_wan.png" showed that PPPoE interface was created and assigned to WAN after configuring it.
Anyhow, this is the current assignment:
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=27890)
Also, the fiber is connected to the SFP0 interface, but it always shows the status of "no carrier". I'm not sure why.
Does that looks to be correct?
Yes, the interface configuration does look correct like so.
But if the interface status is "no carrier", something seems to be wrong on the fiber side in the first place. Or maybe the SFP module is not recognized. I use a ZyXEL GPON SFP in my DEC 750 and when the fiber connection is online, I see this:
(https://i.postimg.cc/dkhDmkzb/2023-06-03-18-06-34-Overview-Interfaces-OPNsense-mgsoft-Mozilla-Firefox.png) (https://postimg.cc/dkhDmkzb)
Some ISPs need the GPON serial, some also check more than that. It is always easier to rule out a misconfiguration when you eliminate factors first. My ISP provided a HUWAEI ONT with an ethernet port, so I tried that first. On those things, you can actually see the GPON status. With many SFP adapters, you have a web interface where you can see this as well.
In order to do this, you have to configure your mxlen0 to a static IP that matches the subnet of the SFP IP address. If you want to access the web interface, you have to create an outbound NAT rule, because there are no configurable routes on the SFP, so you need to "hide" behind your mxlen0 IP.
Then, you can access your SFP web interface. Mine shows this:
(https://i.postimg.cc/pmPS9DjL/2023-06-03-18-20-29-GPON-Mozilla-Firefox.png) (https://postimg.cc/pmPS9DjL)
BTW: Lantiq-based ONT SFP modules like the ZyXEL only show an "up" status for the "ethernet" and allow web access if the fiber is attached and working. So there are three levels of connectivity: 1. fiber connection in order to be able to access the interface, 2. PLOAM status "up" for GPON connectivity (S/N and PLOAM password O.K.) and 3. Correct PPPoE connection via VLAN after authorization (some ISPs use DHCP at this level).
To even achieve a working GPON status was quite hard, as my ISP had a firmware on his ONUs that only worked for HUAWEI non-SFP ONTs - they actually checked the hardware and even the firmware version of the ONTs!
Only after they did some magic on their part was I able to connect with any ONT I wanted (the ISP-provided HUAWEI HG8012, another HUAWEI HG8010H, a ZyXEL PM3000-D20B SFP and a HUAWEI MA5671A SFP). This is possible because I can change the S/N on any of my own devices.
The ISP's fiber is definitely working and I'm connected through it.
When I connect the fiber to the SFP module, it does not show an up status. How can I make sure the module is detected?
Yes, my ISP requires a GPON serial number and I have it. In fact, my current ONU was configured with it.
So, are you saying that I should configure this serial number in the SFP module itself?
Of course, if that is the only S/N that your ISP accepts. Either you copy the provider-supplied ONTs native S/N into your own equipment or you tell your ISP your S/N in order to unlock it (if they accept that - many do not). If you have done neither, this explains a lot, because in both cases, the GPON PLOAM status will never get "up" because the S/N does not match.
And as I wrote, even if the S/N matches, there may be more obstacles like the ISP not accepting anything but their own brand of ONTs or a need for a specific firmware version.
The process of setting the PLOAM password (if needed) and the S/N differs a lot between different brands of SFP ONTs. Some have a web GUI, some have telnet access. Some need a combination, e.g. my ZyXEL shows both on the web GUI:
(https://i.postimg.cc/0r8r66yy/2023-06-04-16-51-48-GPON-Mozilla-Firefox.png) (https://postimg.cc/0r8r66yy)
But only the PLOAM password (called "SLID" here) is changeable in the GUI. The procedure to set the S/N is via telnet, see: https://github.com/xvzf/zyxel-gpon-sfp. You can see I actually chose a S/N that would normally be a HUAWEI one ("HWTC...."). That is why I can use the original ONT as a backup - both devices have the same S/N (most ISPs will only register one S/N).
I see what you mean. My current ONU is a TP-Link XZ000-G3 and I'm able to input the GPON SN and password. I haven't had any issues with it so far.
The PC SFP module is a Miljet MJ-SFPGE-BXD-20S and I could not find any information about its web interface or any configuration of it.
It seems that the module is not a GPON SFP after all. So, there is no GPON SN to configure on it.
At least, I found this interesting source of ONT "hacks", https://hack-gpon.github.io/.
Now, I'll look for the correct module.
Thank you for your insights.
You are mixing incompatible technologies: The Miljet module just has the same single mode fiber as a physical transport as the TP-Link, even the TX wavelengths differ (i.e. the LED light colour is different). What is far worse is that the Miljet is not a GPON SFP module.
Somehow this is like connection a water hose to an gas outlet: It may physically fit and both transport "fluids", but despite that, it will not work. Or to illustrate it:
(https://etel-tuning.eu/wp-content/uploads/2014/10/486-Adapter-Drehstrom-auf-Gardena-510x510.jpg) (https://etel-tuning.eu/produkt/adapter-drehstrom-auf-gardena/)
You have realized by now that this is not an OpnSense problem. We are now at least two layers apart in terms of the OSI model. The hack gpon site you found is a good resource to find info about suitable modules whose S/N can be changed easily. These also differ in the plugs used. SC is the most widely deployed type and it is used in your ZyXEL, but I could not find if it is SC/APC (green plug) or SC/UPC (blue plug) (see https://support.zyxel.eu/hc/de/articles/360005173700-Beschreibung-der-in-GPON-Ger%C3%A4ten-verwendeten-optischen-Anschl%C3%BCsse-SC-APC-und-SC-UPC for the difference). You should look for a GPON SFP module that is compatible with your optical cabling - often, they are available in both flavours.
But remember: Having a real GPON module is a necessary but not necessarily sufficient precondition for this to work - alas your ISP may have other hurdles in stock (mine did!).
I've managed to acquire a GBIC GPON OLT (https://pt.aliexpress.com/item/1005003515662920.html?algo_exp_id=aa53be2c-99d4-4259-ac7b-f8fddfdb03c4-0) that should have a Web interface with the address 192.168.1.1, but I couldn't connect to the module yet. The module is the ODI Realtek DFP-34X-2C2 (https://hack-gpon.org/ont-odi-realtek-dfp-34x-2c2/).
I'm not sure how to make the necessary outbound NAT rule.
The assignments are as follows.
LAN (igc1) -> v4: 192.168.1.6/24
OPT2 (mlxen0) -> v4: 192.168.1.1/24
PC -> 192.168.1.3/24
For now, I've set the outbound NAT as Hybrid and created the following rule, but it doesn't seem to work.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29206)
Any tips?
Change the LAN to 192.168.11.0/24 with the LAN interface 192.168.11.1/24 and disable that rule
Yes, change the LAN network range to anything but 192.168.1.0/24, because otherwise it will be impossible to route any packets from LAN to OPT2, however, you still need the NAT rule from LAN to there, because the ONT does not have a route and only answers to packets from its own network.
Yeah I was pondering that too actually, initially I thought that would be Lan to Wan traffic but NAT seems to be required here after all.
I've changed the LAN network to 192.168.7.0/24.
This is the new configuration.
LAN (igc1) -> v4: 192.168.7.6/24
OPT2 (mlxen0) -> v4: 192.168.1.1/24
PC -> 192.168.7.3 - GW: 192.168.7.6
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29221)
If I understood correctly, this is how the outbound NAT should be:
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29223)
Unfortunately, this way the 192.168.1.1 address only takes me back to OPNSense UI.
Of course it does, since you have set the OPT2 interface address of the OpnSense to 192.168.1.1. So, if you access 192.168.1.1, you reach the OPT2 interface of the OpnSense, never anything else that is connected to it.
If 192.168.1.1 is the designated IP of your ONT, you must set the OPT2 interface to something else in the 192.168.1.0/24 network, say 192.168.1.3. This is basic networking.
Also, it seems like OPT2 state is not "up", so I wonder if there is anything actually connected to that interface, because then there should be no red X.
I see. I've set the OPT2 interface with address 192.168.1.2 now.
Still, I can't access the module. The interface only shows "no carrier" status.
mlxen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: OPT2 (opt2)
inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
status: no carrier
Also, the driver seems to be ok.
root@OPNsense:~ # pciconf -lv mlx4_core0
mlx4_core0@pci0:5:0:0: class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0113
vendor = 'Mellanox Technologies'
device = 'MT27500 Family [ConnectX-3]'
class = network
subclass = ethernet
root@OPNsense:~ # pciconf -a mlx4_core0
mlx4_core0: attached
How can I diagnose that?
Some of the ONTs go online only when they have a valid GPON connect, otherwise they respond only on the serial connection. It may well be that you cannot connect via the network interface as long as there is no GPON online status.
The webpage for that ONT (https://hack-gpon.org/ont-odi-realtek-dfp-34x-2c2/) seems to indicate this in that is explains how to connect via a serial connection and how to set the various parameters.
Some ONT modules need a GPON connection at all (i.e. not a really connected one, just the GPON optical signal) in order to show an online status to the attached computer.
I bet you did not plug in the optical cable yet, did you?
You may have to do this and then access the ONT in order to set the serial # and/or PLOAM to get it fully working.
I did plug the fiber to the module but it wouldn't connect anyway.
Upon monitoring the dashboard, I noticed that the interface was oscillating. Briefly, it seemed to connect before disconnecting again.
During one of these short-lived connections, the interface status briefly showed as "active". I attempted to ping the interface, but unfortunately, I couldn't establish a route to the host. Simultaneously, the host console displayed a "link up" message, followed by a "link down" notification.
After a few reboots, I finally accessed the module. Once I configured an upstream gateway (192.168.1.1) and disabled the outbound NAT, the module was activated. This allowed me to successfully configure the GPON settings.
Now, with the fiber connected, VLAN set, GPON SN and PASS configured, I adjusted the interfaces to better match my environment.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29284)
I also configured the PPPoE interface as WAN, but it stubbornly refused to connect. The PPP logs only showed repeated reconnection attempts.
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1154"] caught fatal signal TERM
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1155"] [wan] IFACE: Close event
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1156"] [wan] IPCP: Close event
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1157"] [wan] IPV6CP: Close event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1158"] [wan] Bundle: Shutdown
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1159"] [wan_link0] Link: Shutdown
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1160"] process 3289 terminated
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1161"] Multi-link PPP daemon for FreeBSD
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1162"]
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1163"] process 53835 started, version 5.9
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1164"] web: web is not running
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1165"] [wan] Bundle: Interface ng0 created
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1166"] [wan_link0] Link: OPEN event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1167"] [wan_link0] LCP: Open event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1168"] [wan_link0] LCP: state change Initial --> Starting
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1169"] [wan_link0] LCP: LayerStart
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1170"] [wan_link0] PPPoE: Connecting to ''
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1171"] [wan_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1172"] [wan_link0] Link: DOWN event
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1173"] [wan_link0] LCP: Down event
The module seems to be receiving the correct signal,
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29282)
yet I'm unsure of the best course of action from here.
Any thoughts?
You are getting there...
1. The ONT's GPON status O5 is fine, you have a GPON connection.
2. There seems to be something wrong with your WAN connection. Have you verified that under interface assigments, WAN is pointing to pppoe0 (xxxxx_vlan600), where xxxxx is the same physical interfaces as your FIB0? Are the pppoe credentials still set up correctly?
You still have to make sure that WAN -> pppoe0 -> xxxxx_vlan600, but last time, xxxxx was mlxen0 and now probably is not any more.
I must be messing up the configuration, but the PPPoE credentials are correct.
These are my current assignments.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29315)
This is the WAN configuration
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29317)
I created a VLAN inteface with tag 600.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29319)
And this is PPPoE interface.
(https://forum.opnsense.org/index.php?action=dlattach;topic=34057.0;attach=29321)
Probably not, since you would see authentication errors if the credentials were wrong, but infact you see timeouts. There is a PPPoE log file, BTW. Also, because the same setup worked / works (?) with an external ONT, I doubt that the credentials are the culprit.
Are you sure that your ISP uses VLAN 600 and PPPoE / PPPoEv6? Many have PPPoE, but DHCPv6.
On the other hand, that seems to have worked with the other ONT as well...
Indeed, the provided credentials are accurate and work successfully on my external ONT (TP-Link XZ000-G3).
In configuring this TP-Link ONT, I focused on setting up the VLAN 600, as well as inputting the GPON SN and Pass, and it works seamlessly in bridge mode.
My current network setup further involves an Asus Router, which connects using PPPoE. It does not require any explicit VLAN configuration. I'm uncertain whether the VLAN setup does not exist in the router or if it is managed automatically, though.
Regarding the PPPoE log, I believe you're referring to the "/var/log/ppps/latest.log" file. This log file corresponds to the data presented under "Interfaces > Point-to-Point > Log File."
It contains only records of connection timeouts.
# tail -f /var/log/ppps/latest.log
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="221"] process 86824 started, version 5.9
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="222"] web: web is not running
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="223"] [wan] Bundle: Interface ng0 created
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="224"] [wan_link0] Link: OPEN event
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="225"] [wan_link0] LCP: Open event
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="226"] [wan_link0] LCP: state change Initial --> Starting
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="227"] [wan_link0] LCP: LayerStart
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="228"] [wan_link0] PPPoE: Connecting to ''
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="229"] [wan_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="230"] [wan_link0] Link: DOWN event
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="231"] [wan_link0] LCP: Down event
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="232"] [wan_link0] Link: reconnection attempt 1 in 3 seconds
Previously, I configured the WAN interface with PPPoEv6. However, I really don't know if that's what my ISP uses. So, I switched to DHCPv6. Still, there is no successfull authentication with PPPoE.
Did you try without the VLAN?
While doing a packet capture, I noticed that when the PPPoE interface is configured to VLAN600, it only sends PPPoE broadcasts and there is no response at all.
The packets are sent with vlan tag 600.
FIB0 mlxen1 2023-08-26 23:37:40.453668 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype 802.1Q (0x8100), length 40: vlan 600, p 0, ethertype PPPoE D, PPPoE PADI [Host-Uniq 0xC0AB562B01F8FFFF] [Service-Name]
FIB0 mlxen1 2023-08-26 23:37:44.471702 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype 802.1Q (0x8100), length 40: vlan 600, p 0, ethertype PPPoE D, PPPoE PADI [Host-Uniq 0xC0AB562B01F8FFFF] [Service-Name]
When I remove VLAN600 and set PPPoE on the mlxen1 (fiber) interface, it sends PPPoE discoveries with no vlan tag, but there is a remote response from two concentrators with vlan tag 600, and that's it. There is no subsequent PPPoE request from OPNsense.
FIB0 mlxen1 2023-08-26 23:41:51.475540 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype PPPoE D (0x8863), length 36: PPPoE PADI [Host-Uniq 0x80D8092000F8FFFF] [Service-Name]
FIB0 mlxen1 2023-08-26 23:41:51.486668 04:b0:e7:c9:45:73 00:02:c9:9b:77:89 ethertype 802.1Q (0x8100), length 60: vlan 600, p 7, ethertype PPPoE D, PPPoE PADO [Host-Uniq 0x80D8092000F8FFFF] [Service-Name] [AC-Name "ME-BSA4A"]
FIB0 mlxen1 2023-08-26 23:41:51.490523 04:b0:e7:c9:44:38 00:02:c9:9b:77:89 ethertype 802.1Q (0x8100), length 60: vlan 600, p 7, ethertype PPPoE D, PPPoE PADO [Host-Uniq 0x80D8092000F8FFFF] [Service-Name] [AC-Name "ME-BSA4B"]
That seems strange. It is almost as if VLAN tags are added automatically in only one direction (or stripped in one).
Probably, your ISP has some magic VLAN configuration in their provided external ONTs? I know for sure that such constructs exist in Huawei ONTs where you can have different profiles for specific services, like different VLANs.
Over here in germany, ISPs are required to be open w/r to what equipment is used to provide access, namely the network termination must be passive if the customer requires it. However, many providers do not like that specific law and deliberately do their best to maximize hurdles to actually make use of that freedom. Recently, there is a discussion going on to revise the law for reasons of potential disturbance of networks by customers who use non-certified equipment of some sort.
I do not think that there is a general problem with OpnSense or your configuration at this point. You can easily check if there is when you use the ISP-provided ONT with an otherwise unaltered configuration. If that works, either your GPON module has a problem with VLANs or there must be something special in the ISP's ONT.
P.S.: After looking at https://hack-gpon.org/ont-odi-realtek-dfp-34x-2c2/, I see some hints about VLAN problems with your specific GPON module in specific firmware revisions!
See https://github.com/Anime4000/RTL960x/tree/main/Firmware/DFP-34X-2C2, there seem to be some fixes out for those problems. It seems there are firmwares for different modes... I cannot help with this rabbit hole, though, since I have a different ISP and a different GPON module. Maybe it would be advisable to contact the hack gpon folks or to use a different module, preferably one that is known to work with your ISP.
Finally, it works.
The issue turned out to be the module firmware, as you suspected.
Initially, I had concerns that a failed firmware update might render the module inaccessible, especially considering I cannot access its serial console.
Although I had noticed the new firmware versions, I didn't deem it necessary to upgrade immediately. I wanted to ensure that everything was functioning before attempting any updates.
Moreover, given my limited familiarity with optical networks, comprehending the various modes and options presented a bit of a challenge.
The module was initially running version "v1.0_220923". When all else failed, I took the step to update the firmware to the most recent version, "V1.2.2-221209". After configuring the module, voila! OPNsense successfully acquired new public IP addresses. No VLAN configuration was needed by-the-way.
What a journey!
I am immensely grateful for all the insights and tips provided to me. These inputs were incredibly valuable and guided me towards the correct adjustments. The learning I've gained throughout this process has been invaluable.
Thank you.
Now, a new journey begins... optimizing everything. ;D
You're welcome and congrats! That was quite a steep learning curve and you did very well.
I personally find it easy to isolate technical problems when I can check each link in the chain in turn, like OSI layers in this case. Many people do not understand that and give up without even knowing what exactly went wrong.
Admittedly, with GPON technology, there are many things you have to know (or learn) in order to get things working and it takes determination to follow through - which you had.