Messages

Finally, it works.
The issue turned out to be the module firmware, as you suspected.
Initially, I had concerns that a failed firmware update might render the module inaccessible, especially considering I cannot access its serial console.
Although I had noticed the new firmware versions, I didn't deem it necessary to upgrade immediately. I wanted to ensure that everything was functioning before attempting any updates.
Moreover, given my limited familiarity with optical networks, comprehending the various modes and options presented a bit of a challenge.

The module was initially running version "v1.0_220923". When all else failed, I took the step to update the firmware to the most recent version, "V1.2.2-221209". After configuring the module, voila! OPNsense successfully acquired new public IP addresses. No VLAN configuration was needed by-the-way.

What a journey!
I am immensely grateful for all the insights and tips provided to me. These inputs were incredibly valuable and guided me towards the correct adjustments. The learning I've gained throughout this process has been invaluable.
Thank you.

Now, a new journey begins... optimizing everything.  ;D

While doing a packet capture, I noticed that when the PPPoE interface is configured to VLAN600, it only sends PPPoE broadcasts and there is no response at all.
The packets are sent with vlan tag 600.

Code Select Expand

FIB0 mlxen1 2023-08-26 23:37:40.453668 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype 802.1Q (0x8100), length 40: vlan 600, p 0, ethertype PPPoE D, PPPoE PADI [Host-Uniq 0xC0AB562B01F8FFFF] [Service-Name]
FIB0 mlxen1 2023-08-26 23:37:44.471702 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype 802.1Q (0x8100), length 40: vlan 600, p 0, ethertype PPPoE D, PPPoE PADI [Host-Uniq 0xC0AB562B01F8FFFF] [Service-Name]

When I remove VLAN600 and set PPPoE on the mlxen1 (fiber) interface, it sends PPPoE discoveries with no vlan tag, but there is a remote response from two concentrators with vlan tag 600, and that's it. There is no subsequent PPPoE request from OPNsense.

Code Select Expand

FIB0 mlxen1 2023-08-26 23:41:51.475540 00:02:c9:9b:77:89 ff:ff:ff:ff:ff:ff ethertype PPPoE D (0x8863), length 36: PPPoE PADI [Host-Uniq 0x80D8092000F8FFFF] [Service-Name]
FIB0 mlxen1 2023-08-26 23:41:51.486668 04:b0:e7:c9:45:73 00:02:c9:9b:77:89 ethertype 802.1Q (0x8100), length 60: vlan 600, p 7, ethertype PPPoE D, PPPoE PADO [Host-Uniq 0x80D8092000F8FFFF] [Service-Name] [AC-Name "ME-BSA4A"]
FIB0 mlxen1 2023-08-26 23:41:51.490523 04:b0:e7:c9:44:38 00:02:c9:9b:77:89 ethertype 802.1Q (0x8100), length 60: vlan 600, p 7, ethertype PPPoE D, PPPoE PADO [Host-Uniq 0x80D8092000F8FFFF] [Service-Name] [AC-Name "ME-BSA4B"]

Indeed, the provided credentials are accurate and work successfully on my external ONT (TP-Link XZ000-G3).
In configuring this TP-Link ONT, I focused on setting up the VLAN 600, as well as inputting the GPON SN and Pass, and it works seamlessly in bridge mode.
My current network setup further involves an Asus Router, which connects using PPPoE. It does not require any explicit VLAN configuration. I'm uncertain whether the VLAN setup does not exist in the router or if it is managed automatically, though.

Regarding the PPPoE log, I believe you're referring to the "/var/log/ppps/latest.log" file. This log file corresponds to the data presented under "Interfaces > Point-to-Point > Log File."
It contains only records of connection timeouts.

Code Select Expand

# tail -f /var/log/ppps/latest.log
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="221"] process 86824 started, version 5.9
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="222"] web: web is not running
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="223"] [wan] Bundle: Interface ng0 created
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="224"] [wan_link0] Link: OPEN event
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="225"] [wan_link0] LCP: Open event
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="226"] [wan_link0] LCP: state change Initial --> Starting
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="227"] [wan_link0] LCP: LayerStart
<30>1 2023-08-25T17:15:59-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="228"] [wan_link0] PPPoE: Connecting to ''
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="229"] [wan_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="230"] [wan_link0] Link: DOWN event
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="231"] [wan_link0] LCP: Down event
<30>1 2023-08-25T17:16:08-03:00 OPNsense.domain ppp 86824 - [meta sequenceId="232"] [wan_link0] Link: reconnection attempt 1 in 3 seconds

Previously, I configured the WAN interface with PPPoEv6. However, I really don't know if that's what my ISP uses. So, I switched to DHCPv6. Still, there is no successfull authentication with PPPoE.

I must be messing up the configuration, but the PPPoE credentials are correct.
These are my current assignments.


This is the WAN configuration


I created a VLAN inteface with tag 600.


And this is PPPoE interface.

[I also configured the PPPoE interface as WAN, but it stubbornly refused to connect. The PPP logs only showed repeated reconnection attempts.]

I did plug the fiber to the module but it wouldn't connect anyway.

Upon monitoring the dashboard, I noticed that the interface was oscillating. Briefly, it seemed to connect before disconnecting again.
During one of these short-lived connections, the interface status briefly showed as "active". I attempted to ping the interface, but unfortunately, I couldn't establish a route to the host. Simultaneously, the host console displayed a "link up" message, followed by a "link down" notification.

After a few reboots, I finally accessed the module. Once I configured an upstream gateway (192.168.1.1) and disabled the outbound NAT, the module was activated. This allowed me to successfully configure the GPON settings.

Now, with the fiber connected, VLAN set, GPON SN and PASS configured, I adjusted the interfaces to better match my environment.


I also configured the PPPoE interface as WAN, but it stubbornly refused to connect. The PPP logs only showed repeated reconnection attempts.

Code Select Expand

<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1154"] caught fatal signal TERM
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1155"] [wan] IFACE: Close event
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1156"] [wan] IPCP: Close event
<30>1 2023-08-23T17:36:04-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1157"] [wan] IPV6CP: Close event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1158"] [wan] Bundle: Shutdown
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1159"] [wan_link0] Link: Shutdown
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 3289 - [meta sequenceId="1160"] process 3289 terminated
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1161"] Multi-link PPP daemon for FreeBSD
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1162"]
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1163"] process 53835 started, version 5.9
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1164"] web: web is not running
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1165"] [wan] Bundle: Interface ng0 created
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1166"] [wan_link0] Link: OPEN event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1167"] [wan_link0] LCP: Open event
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1168"] [wan_link0] LCP: state change Initial --> Starting
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1169"] [wan_link0] LCP: LayerStart
<30>1 2023-08-23T17:36:06-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1170"] [wan_link0] PPPoE: Connecting to ''
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1171"] [wan_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1172"] [wan_link0] Link: DOWN event
<30>1 2023-08-23T17:36:15-03:00 OPNsense.domain ppp 53835 - [meta sequenceId="1173"] [wan_link0] LCP: Down event


The module seems to be receiving the correct signal,


yet I'm unsure of the best course of action from here.
Any thoughts?

I see. I've set the OPT2 interface with address 192.168.1.2 now.
Still, I can't access the module. The interface only shows "no carrier" status.

Code Select Expand

mlxen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT2 (opt2)
        inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
        status: no carrier

Also, the driver seems to be ok.

Code Select Expand

root@OPNsense:~ # pciconf -lv mlx4_core0
mlx4_core0@pci0:5:0:0:  class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0113
    vendor     = 'Mellanox Technologies'
    device     = 'MT27500 Family [ConnectX-3]'
    class      = network
    subclass   = ethernet

root@OPNsense:~ # pciconf -a mlx4_core0
mlx4_core0: attached

How can I diagnose that?

I've changed the LAN network to 192.168.7.0/24.
This is the new configuration.

Code Select Expand

LAN (igc1)      -> v4: 192.168.7.6/24
OPT2 (mlxen0)   -> v4: 192.168.1.1/24
PC -> 192.168.7.3 - GW: 192.168.7.6


If I understood correctly, this is how the outbound NAT should be:


Unfortunately, this way the 192.168.1.1 address only takes me back to OPNSense UI.

I've managed to acquire a GBIC GPON OLT that should have a Web interface with the address 192.168.1.1, but I couldn't connect to the module yet. The module is the ODI Realtek DFP-34X-2C2.
I'm not sure how to make the necessary outbound NAT rule.

The assignments are as follows.
LAN (igc1)      -> v4: 192.168.1.6/24
OPT2 (mlxen0)   -> v4: 192.168.1.1/24
PC -> 192.168.1.3/24

For now, I've set the outbound NAT as Hybrid and created the following rule, but it doesn't seem to work.


Any tips?

I see what you mean. My current ONU is a TP-Link XZ000-G3 and I'm able to input the GPON SN and password. I haven't had any issues with it so far.

The PC SFP module is a Miljet MJ-SFPGE-BXD-20S and I could not find any information about its web interface or any configuration of it.
It seems that the module is not a GPON SFP after all. So, there is no GPON SN to configure on it.

At least, I found this interesting source of ONT "hacks", https://hack-gpon.github.io/.

Now, I'll look for the correct module.
Thank you for your insights.

The ISP's fiber is definitely working and I'm connected through it.
When I connect the fiber to the SFP module, it does not show an up status. How can I make sure the module is detected?

Yes, my ISP requires a GPON serial number and I have it. In fact, my current ONU was configured with it.
So, are you saying that I should configure this serial number in the SFP module itself?

But, I thought picture "pppoe_wan.png" showed that PPPoE interface was created and assigned to WAN after configuring it.

Anyhow, this is the current assignment:


Also, the fiber is connected to the SFP0 interface, but it always shows the status of "no carrier". I'm not sure why.

Does that looks to be correct?

I'm sorry, but I really don't get it.
As you can see from the previous pictures, VLAN is assigned to "mlxen0" as the parent interface.
And WAN interface is assinged to PPPoE. When I configured the "configuration type" to PPPoE, it automatically switched the assingment to PPPoE.

I've reset everything and tried again.

Here's what it looks like now:

Code Select Expand

LAN (igc0)      -> v4: 192.168.7.222/24
OPT1 (igc1)     ->
OPT2 (igc2)     ->
SFP0 (mlxen0_vlan600) ->
SFP1 (mlxen1)   ->
WAN (pppoe0)    ->

I have created a VLAN with id 600.


Then, I set up the VLAN as the WAN interface.


Finally, I configured the WAN interface to use PPPoE.


After applying that configuration, a PPPoE interface was created.


The WAN shows as UP on the Dashboard, but there's no Internet connection available.

My PC uses the "Mellanox ConnectX-3" controller which is listed as a supported network interface.

Code Select Expand

# pciconf -lv mlx4_core0
mlx4_core0@pci0:5:0:0:  class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0113
    vendor     = 'Mellanox Technologies'
    device     = 'MT27500 Family [ConnectX-3]'
    class      = network
    subclass   = ethernet

I did install OPNsense, but I'm having a hard time finding information about how to setup SFP with PPPoE.

So far, I have configured the SFP interface type as PPPoE.

After connecting the fiber to the interface, the logs show that the PPP interface is unable to connect.

Code Select Expand

<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="441"] [opt5_link0] Link: reconnection attempt 72
<30>1 2023-05-17T19:58:24-03:00 OPNsense ppp 76831 - [meta sequenceId="442"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="443"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="444"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="445"] [opt5_link0] LCP: Down event
<30>1 2023-05-17T19:58:33-03:00 OPNsense ppp 76831 - [meta sequenceId="446"] [opt5_link0] Link: reconnection attempt 73 in 3 seconds
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="447"] [opt5_link0] Link: reconnection attempt 73
<30>1 2023-05-17T19:58:36-03:00 OPNsense ppp 76831 - [meta sequenceId="448"] [opt5_link0] PPPoE: Connecting to ''
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="449"] [opt5_link0] PPPoE connection timeout after 9 seconds
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="450"] [opt5_link0] Link: DOWN event
<30>1 2023-05-17T19:58:45-03:00 OPNsense ppp 76831 - [meta sequenceId="451"] [opt5_link0] LCP: Down event

I am aware that my ISP's ONU uses a specific GPON SN and a VLAN id, but I don't know how to setup them on OPNsense.

My current setup uses an ONU to bridge the ISPs fiber to my SOHO router, which handles the PPPoE connection.
Additionally, I had to input the GPON SN and the VLAN ID to connect the ONU.

I have a mini PC with an SFP+ port and I want to plug it directly into my ISP's fiber. However, I am not very familiar with OPNsense and I'm unsure if that is feasible.

Do you have any tips or tutorials to share regarding this?