PPPoE over VLAN through managed switch doesn't work (but direct connection does)

[Salty72]

Hi,
Happy OpnSense user here.
I've run into an issue (which is probably not a OpnSense issue) but I thought I would ask in case I missed a setting or someone has resolved this issue in the past.

Background:
My OpnSense is configured to initiate a PPPoE connection through an interface (igb1) and it works perfectly.
OpnSense (untagged) -> Modem (VLAN 35) -> ISP (this works)

Issue:
I want to eventually separate the location of the modem and OpnSense using a trunk to initiate the PPPoE connection.
So I tested as follows:
1. Ensure modem can leave VLAN untouched by tagging the packets in OpnSense. (this works)
2. Pass the tagged packets though a managed switch to ensure the switch si not creating issues. (I'm failing here)
3. Send the tagged packets through a trunk to another switch where the modem will reside.

Step 1:
To ensure that the modem can support leaving the VLAN untouched, I switched the PPPoE connection to use VLAN 35, change my modem setting to leave the VLAN unchanged and everything appears to work correctly
OpnSense (VLAN 35) -> Modem (VLAN untouched) -> ISP (this works)

Based on the above, it appears that the PPPoE connection seems to work well with untagged or tagged (35) packets when OpnSense and the modem are directly connected.

Step 2:
I tried sending OpnSense (VLAN 35) -> (VLAN 35) switch (VLAN 35) -> modem (VLAN untouched) -> ISP but I keep getting connection timed out.
Both ports on switch are members of VLAN 35; no other ports are members of VLAN 35.

Step 2a:
Then I tried OpnSense (untagged) -> (PVID 35) switch (VLAN 35) -> modem -> ISP but I also keep getting connection timed out.
No other ports are members of VLAN 35.

Step 2c:
I thought maybe the switch (Brocade TurboIron) was doing something odd (or had a bug) so I tried another brand (Fastpath) and got the same timeout results.

Is there a setting that I'm missing in OpnSense or the managed switches to allow PPPoE over VLAN to work when going through a managed switch?

Thanks in advance,
Sal

[Patrick M. Hausen]

You need

OPNsense (tagged) VLAN 35 --> Trunk Port in Switch (all VLANs tagged) --> Trunk Port on Switch (all VLANs tagged) --> Modem

If you assign a switch port to a VLAN that means untagged.

[Salty72]

In the switch, I can apply ports to the VLANs as tagged or untagged. I applied both ports as tagged to VLAN 35.
Did not work.

I also took your suggestion to tag all the VLANs to the port (essentially making it a trunk) and experienced the same timeout failure.
OPNsense (tagged) VLAN 35 --> Trunk Port in Switch (all VLANs tagged) --> Trunk Port on Switch (all VLANs tagged) --> Modem

Any other ideas would be appreciated.

Thanks,

[Patrick M. Hausen]

Sorry, strictly Cisco or Juniper gear here - in which a port is either a "trunk" or an "access port".

Why not try this - or did you already?

OPNsense --> phys. Interface (untagged) w. PPPoE --> switch access port (untagged) VLAN 35 ---> switch trunk port (tagged) --> modem

HTH,
Patrick

[Salty72]

I tried it as the second attempt (2a) with the same result (no response).
OpnSense (untagged) -> (PVID 35) switch (VLAN 35) -> modem -> ISP

I know a direct connection works for both scenarios below:
1. Opnsense sending the packet as VLAN 35 with the modem passing the packet untouched
2. OpnSense sending the packet untagged with the modem retagging it as VLAN 35.
 
Whenever I include any switch in the path, it fails.
1. Passthrough VLAN 35 from OpnSense to modem.
2. Take VLAN 35 packet from OpnSense and clear the tag before sending to the modem (and let the modem tag it for the ISP).
3. Take untagged packet from OpnSense and tag it as VLAN 35 before sending it to the modem.

I'm out of ideas so please keep any suggestions coming.
I'm will to try them all.

Maybe I will try sending an untagged packet from OpnSense to the switch to the modem.
Problem is that this will not help me traverse the packet through a trunk.
I need the packet tagged when it leaves OpnSense.

[Patrick M. Hausen]

And why can't you use OPNsense untagged --> port based VLAN X in whatever switch infrastructure (need not even be 35) --> still port based egress (untagged) --> modem (adds VLAN 35)?

[Salty72]

Tried it and it still didn't work.
OPNsense untagged --> port based VLAN X in whatever switch infrastructure (need not even be 35) --> still port based egress (untagged) --> modem (adds VLAN 35)

But I found the solution.
I decided to just try changing different port settings on the modem (didn't expect any effect) and it worked when I forced the speed (100Mb/s) and duplex setting (both).
No idea why since I never had an issue with the switches or the modem auto-negotiating before (except when connected to each other obviously even though it appears to auto-negotiate correctly based on the switch and modem status page).
Hope this will help someone in the future if the encounter this weird behavior.
Any idea how I can update the thread as solved?