OPNsense not connecting to WAN

[OPNonsense]

Hello,


It's me again -- the guy who perennially and rather persistently is failing to install OPNsense on his home network.

I previously posted here a couple of days ago: https://forum.opnsense.org/index.php?topic=35368.0

As things stand, I have learnt some things, had some minor successes and insights, but sadly the end result is I am still unable to get my WAN connected via the OPNsense box·

Summary of what I have done in as few words as I can put it.


MY SET-UP
- My current ISP-issued router is the  FRITZ!Box 7530
- Its internal network is 192.168.178.x/24
- My OPNsense box is an AOPEN DE3250 mini PC (Intel N2930) with 2 x NICs
- All LAN traffic is routed through a Netgear GS234T smart switch


WHAT I HAVE DONE SINCE LAST TIME
- Reset the OPNsense box to default
- Configured interfaces, set LAN to 192.168.10.1
- Web interface working fine
- Then put FRITZ!Box in bridge mode ("Use FRITZ!Box as an IP client") and left it switched off for a second while I configured the WAN interface
- Set WAN to DHCP, leaving it on the 192.168.178.x/24 subnet, same as the FRITZ!Box
- Ensured that WAN was set up on OPNsense box for DHCP
- Plugged in FRITZ!Box directly into the OPNsense WAN port that I had configured at the beginning
- Reboot both devices
- OPNsense comes online, shows no IP address assigned to the FRITZ!Box (0.0.0.0/8)
- Try to play around with firewall rules, allowing all traffic to and from NAT
- Also previously already enabled my normal DHCP server (Pihole) and used OPNsense's
- Both tried to use my Pihole DNS and Unbound DNS
- Tried to configure some outbound NAT rules
- There was some talk about me being on double NAT, but it would appear that I am not; way I checked was by checking what the FRITZ!Box said was my public IP, comparing that to one of those "what's my IP?" sites, and doing a traceroute as well, but could not see any discrepancy there. If I did this wrong, let me know
- Loads of back-and-forth between the above options, endless tweaking, restarting, trying to use FRITZ!Box as normal router via OPNsense, trying to toy around with PPPoE and failing so miserably I nearly locked myself out of the FRITZ!Box etc.
- Here is the odd thing, though: When connected directly to the OPNsense box (not the web interface, but the CLI interface), I was briefly able to ping www.google.com, which tells me that there was some kind of WAN connection in place. I have not been able to replicate this
- I watched many videos and read guides, but ultimately, everyone's set-up is different or everything magically slots into place for everyone else

Any ideas at all? I won't give up until this is done.

[Seimus]

Let me ask here,

So to sum it up >
1. You put FritzBox into bridge mode instead route mode
2. You connect OPNsense to the FritzBOX, and configure on it PPPOE over WAN and configure WAN for DHCP to receive IP from the ISP
3. You set LAN Staticly, DHCP server is Pihole

The outcome is that you can not receive DHCP IP on the WAN as well no host on the LAN is receiving IP? Or its only WAN problem?
Also what do you mean by this "Set WAN to DHCP, leaving it on the 192.168.178.x/24 subnet, same as the FRITZ!Box" ?

When you set DHCP you dont set any Subnet range as the ISP DHCP will announce it to you.

When you do a fresh install of OPN without any changes of rules, and let NAT on auto, settings WAN as DHCP + set your PPPoE connection over WAN. OPNsense should work without problem aka the reachability to internet should work "out of the box"

Regards,
S.

[Hydranet]

I don't know if it helps it sounds like your issue is similar to mine, as in your WAN interface not getting an ip from you ISP modem which is set in bridge mode. What was the case for me I had my LAN plugged in port1 and WAN plugged in port2 (which is the default)and then I didn't get a WAN IP from dhcp. What I ended up doing on making it work was change the interfaces assignment by using either ssh or a keyboard and a monitor to connect to OPNsense. Then I selected option 1, I then assigned igc0 as the WAN interface and igc1 as the LAN interface, after which my WAN interface was succesfully able to get an ip from dhcp. I don't know if it will work for you but I thought it would be worth mentioning for you to try.

[OPNonsense]

Let me ask here,

So to sum it up >
1. You put FritzBox into bridge mode instead route mode
2. You connect OPNsense to the FritzBOX, and configure on it PPPOE over WAN and configure WAN for DHCP to receive IP from the ISP
3. You set LAN Staticly, DHCP server is Pihole

The outcome is that you can not receive DHCP IP on the WAN as well no host on the LAN is receiving IP? Or its only WAN problem?
Also what do you mean by this "Set WAN to DHCP, leaving it on the 192.168.178.x/24 subnet, same as the FRITZ!Box" ?

When you set DHCP you dont set any Subnet range as the ISP DHCP will announce it to you.

When you do a fresh install of OPN without any changes of rules, and let NAT on auto, settings WAN as DHCP + set your PPPoE connection over WAN. OPNsense should work without problem aka the reachability to internet should work "out of the box"

Regards,
S.

Sorry for being unclear.


Well, this is the thing. I am not sure on the FRITZ!Box settings. What I selected was "Use this FRITZ!Box as an IP device" with a text saying that this means routing will be done by another device on the network.

I didn't configure any PPPoE stuff -- should I have?

The problem appears to be WAN only. Pihole as a DHCP server has been switched off to be on the safe side (I do this every time before I install OPNsense, and active OPNsense's DHCP server).

Are you saying I should just leave Pihole as the DHCP server for the WAN interface? I am not sure on the PPPoE stuff and may need to read up on that. Thanks for responding.

As regards your comment here:

Also what do you mean by this "Set WAN to DHCP, leaving it on the 192.168.178.x/24 subnet, same as the FRITZ!Box" ?

What I meant by that is that I left the WAN interface on the same subnet that the FRITZ!Box natively is. This was so that I wouldn't have to put the FRITZ!Box on a different subnet as I was told it may only be configured to route on that particular subnet.

[OPNonsense]

I don't know if it helps it sounds like your issue is similar to mine, as in your WAN interface not getting an ip from you ISP modem which is set in bridge mode. What was the case for me I had my LAN plugged in port1 and WAN plugged in port2 (which is the default)and then I didn't get a WAN IP from dhcp. What I ended up doing on making it work was change the interfaces assignment by using either ssh or a keyboard and a monitor to connect to OPNsense. Then I selected option 1, I then assigned igc0 as the WAN interface and igc1 as the LAN interface, after which my WAN interface was succesfully able to get an ip from dhcp. I don't know if it will work for you but I thought it would be worth mentioning for you to try.

How strange! It's weird that that would work. But I will bear that in mind and will give that a go on the next install. Sadly cannot try this now before work (night shift) as my wife is working from home today and I cannot break the internet, but will give this a go tomorrow!

[Hydranet]

How strange! It's weird that that would work. But I will bear that in mind and will give that a go on the next install. Sadly cannot try this now before work (night shift) as my wife is working from home today and I cannot break the internet, but will give this a go tomorrow!

The only logical explanation I can think of is that for some reason even though the physical port2 being connected to the ISP modem in bridge mode is actually seen as logical port1(igc0) in the os. I also tried manually going through option 2(Set interface IP address) and setting the LAN interface with a static ip and the WAN interface with dhcp but that didn't make dhcp give my wan interface a WAN ip either. I hope it works for you!

[Seimus]

Alright so maybe we are coming here possible conclusion.

Just let me verify one last thing if its the yes the bellow will be valid. And lets focus for now only on the WAN.

Your ISP delivers you the Internet connection via PPPoE and FritzBox is from the ISP as the Router/xDSL modem correct? (Also you are located in Germany? that would mean the owner of the underlay xDSL infra is DT...).


Each xDSL router/modem has two modes > Routed mode and Bridge mode.

Routed mode
The devices act as xDSL modem as well as router. It does establish xDSL connection and on top of it PPPoE for registration. Further it does routing. This means such a device does all what it needs from L1 till L3.

Bridge mode
This device act only as a xDSL modem. Meaning it doesn't care about PPPoE or L3. its only purpose its to properly encapsulate packet/frames within xDSL. Within this mode you need to have another device capable PPPoE and L3 where you need to configure PPPoE in order to establish the connectivity towards DSLAM.


What I understand what you tried to do

• You changed the mode on FritzBox from routed to bridged > "Well, this is the thing. I am not sure on the FRITZ!Box settings. What I selected was "Use this FRITZ!Box as an IP device" with a text saying that this means routing will be done by another device on the network."
• You configured WAN side of the OPN with the IP from the LAN Subnet of FrtizBox when it was in Routed mode
• You try to ping trace or do any other heck activity from the OPN but you are unable to reach the internet like 1.1.1.1 for example.

I can tell you this will not work due to as mentioned of the above. In the moment you set a bridge mode. L3 functionally is not working, only the aka management will work for you in order to connected to the FritzBox but it will not route any traffic towards internet. Because you dont have PPPoE established and because it acts as a modem not as a router. If you want this setup you need to let FrtizBox in the Routed mode but you will hit double NAT. it will still work but dont do it. Rather do the bellow

If you understand the above for Routed and Bridge mode you need to do following:

• Set FrtizBox to bridge mode - from now on it will only work as xDSL modem - https://youtu.be/XE8bj9G6LB4
• Set OPNsence as your PPPoE gateway - basicaly here you will create a new Interface PPPoE and configure the PPPoE same as was on fritz BOX. You will let DHCP on the OPN interface without any static IP assignment. The PPPoE interface needs to be a child of the physical interface that is connected towards FRITZbOX. This can be done under Interface configuration you can set the WAN interface into PPPoE mode and fill the neccessary fields like account/password - VPI/VCI - Annex - etc.

Effectively what will happen you do this is that > FrtizBOx will handle xDSL as modem and OPN will handle PPPoE authentication and encapsulation. This is the setup you want to do. Because multipurpose modes/routers are crap. If you have possibility to do PPPoE on another device always set the ISP device into bridge mode.

Regards,
S.

[rfox]

Just a small hint:  Fritzboxes do NOT have the ability to bridge their modems - the mode setting for the FritzBox as an IP-Client disables the FW & NAT and allows the Fritzbox to be a simple IP client on an existing network (meaning using something else as gateway & firewall)

Here's a reference in English:
https://en.easybell.de/help/telephone-configuration/fritzbox-telephony/using-a-fritzbox-as-ip-client/

If you have a DSL line from your provider - best solution is to invest in a simple DSL Modem like Draytek Vigor130 or Vigor167 - as alternative, you can use a device like the Deutsche Telekom's Speedport Smart 3/4 devices which DO allow bridged mode (modem only)

Then you can use the OPN to get to the internet 

Hope this is helpful . . .

Cheers,
R.Fox

PS:  If you are in Germany and understand German - Here's a good video which shows how to use the OPNSense behind a Fritzbox using exposed host:  https://www.youtube.com/watch?v=-7G6MSVmdn8

[Seimus]

Just a small hint:  Fritzboxes do NOT have the ability to bridge their modems - the mode setting for the FritzBox as an IP-Client disables the FW & NAT and allows the Fritzbox to be a simple IP client on an existing network (meaning using something else as gateway & firewall)

Here's a reference in English:
https://en.easybell.de/help/telephone-configuration/fritzbox-telephony/using-a-fritzbox-as-ip-client/

If you have a DSL line from your provider - best solution is to invest in a simple DSL Modem like Draytek Vigor130 or Vigor167 - as alternative, you can use a device like the Deutsche Telekom's Speedport Smart 3/4 devices which DO allow bridged mode (modem only)

Then you can use the OPN to get to the internet 

Hope this is helpful . . .

Cheers,
R.Fox

hah this is a good one. In that case I agree what is said here, get an only modem capable device.

BTW I found few topics where some people were able to force FrtizBox into bridge mode by adjusting the .conf file

https://theincrowdvlog-com.ngontinh24.com/article/how-do-i-configure-my-fritz-box-in-bridge-mode


Regards,
S.

[rfox]

Just a small hint:  Fritzboxes do NOT have the ability to bridge their modems - the mode setting for the FritzBox as an IP-Client disables the FW & NAT and allows the Fritzbox to be a simple IP client on an existing network (meaning using something else as gateway & firewall)

Here's a reference in English:
https://en.easybell.de/help/telephone-configuration/fritzbox-telephony/using-a-fritzbox-as-ip-client/

If you have a DSL line from your provider - best solution is to invest in a simple DSL Modem like Draytek Vigor130 or Vigor167 - as alternative, you can use a device like the Deutsche Telekom's Speedport Smart 3/4 devices which DO allow bridged mode (modem only)

Then you can use the OPN to get to the internet 

Hope this is helpful . . .

Cheers,
R.Fox

hah this is a good one. In that case I agree what is said here, get an only modem capable device.

BTW I found few topics where some people were able to force FrtizBox into bridge mode by adjusting the .conf file

https://theincrowdvlog-com.ngontinh24.com/article/how-do-i-configure-my-fritz-box-in-bridge-mode


Regards,
S.

As far as I know - this only applies to the Cable modem Fritzboxes (6000 series) and not the DSL ones 

Cheers,
R.Fox

[Seimus]

As far as I know - this only applies to the Cable modem Fritzboxes (6000 series) and not the DSL ones 

Cheers,
R.Fox

That feels like a ripoff..... Thanks for sharing this. I personally avoid FrtizBoxes or per say any OEM and Telco provided device, as you never know what catches it may bring.


Anyway If this is the case for our OP of this thread, if really he is not able to put the BOX into Bride mode only way how to have this worked out with his current setup is to have the FrtizBox in Routed mode, OPN WAN set on Static IP from the LAN Pool FrtizBOX provides and disable on OPN NAT to avoid Double NAT situation. Additionally I would disable FW on the FrtizBox.



The best option would be really as you said "Get a normal xDSL modem and put it into bridge mode".

My personal opinion is > network device on which you dont have control should not be in your home network.

Regards,
S.

[OPNonsense]

Alright so maybe we are coming here possible conclusion.

Just let me verify one last thing if its the yes the bellow will be valid. And lets focus for now only on the WAN.

Your ISP delivers you the Internet connection via PPPoE and FritzBox is from the ISP as the Router/xDSL modem correct? (Also you are located in Germany? that would mean the owner of the underlay xDSL infra is DT...).


Each xDSL router/modem has two modes > Routed mode and Bridge mode.

Routed mode
The devices act as xDSL modem as well as router. It does establish xDSL connection and on top of it PPPoE for registration. Further it does routing. This means such a device does all what it needs from L1 till L3.

Bridge mode
This device act only as a xDSL modem. Meaning it doesn't care about PPPoE or L3. its only purpose its to properly encapsulate packet/frames within xDSL. Within this mode you need to have another device capable PPPoE and L3 where you need to configure PPPoE in order to establish the connectivity towards DSLAM.


What I understand what you tried to do

• You changed the mode on FritzBox from routed to bridged > "Well, this is the thing. I am not sure on the FRITZ!Box settings. What I selected was "Use this FRITZ!Box as an IP device" with a text saying that this means routing will be done by another device on the network."
• You configured WAN side of the OPN with the IP from the LAN Subnet of FrtizBox when it was in Routed mode
• You try to ping trace or do any other heck activity from the OPN but you are unable to reach the internet like 1.1.1.1 for example.

Hey, thanks for such an exhaustive reply.

Despite having a FRITZ!Box as the ISP-issued modem/router combo, I am in fact in the UK. Zen Internet also use FRITZ!Box.

With regards to the two options you presented there, the FRITZ!Box is:

Routed mode
The devices act as xDSL modem as well as router. It does establish xDSL connection and on top of it PPPoE for registration. Further it does routing. This means such a device does all what it needs from L1 till L3.

So yeah, I don't have another device.

The FRITZ!Box gives me two options:


Operating Mode in the Home Network

Here you can change the operating mode of this FRITZ!Box in the home network.
Internet router

This FRITZ!Box is the internet router and makes the internet connection available to the devices in the home network. It makes its own IP address range available, and the firewall remains enabled.

Under Internet > Account Information > Internet Connection you can set how the FRITZ!Box establishes the internet connection.
IP client

This FRITZ!Box extends the home network as an IP client and uses an existing internet connection. It receives an IP address from the router. The FRITZ!Box becomes part of the existing home network and takes over its IP address range, disabling the firewall of the FRITZ!Box.

At the moment, I am obviously using the first option, but whenever I try to set up OPNsense, I set it to the second option, demoting it to an IP device that acts as a modem while having my OPNsense box do the routing.

I can tell you this will not work due to as mentioned of the above. In the moment you set a bridge mode. L3 functionally is not working, only the aka management will work for you in order to connected to the FritzBox but it will not route any traffic towards internet. Because you dont have PPPoE established and because it acts as a modem not as a router. If you want this setup you need to let FrtizBox in the Routed mode but you will hit double NAT. it will still work but dont do it. Rather do the bellow

If you understand the above for Routed and Bridge mode you need to do following:

• Set FrtizBox to bridge mode - from now on it will only work as xDSL modem - https://youtu.be/XE8bj9G6LB4
• Set OPNsence as your PPPoE gateway - basicaly here you will create a new Interface PPPoE and configure the PPPoE same as was on fritz BOX. You will let DHCP on the OPN interface without any static IP assignment. The PPPoE interface needs to be a child of the physical interface that is connected towards FRITZbOX. This can be done under Interface configuration you can set the WAN interface into PPPoE mode and fill the neccessary fields like account/password - VPI/VCI - Annex - etc.

Effectively what will happen you do this is that > FrtizBOx will handle xDSL as modem and OPN will handle PPPoE authentication and encapsulation. This is the setup you want to do. Because multipurpose modes/routers are crap. If you have possibility to do PPPoE on another device always set the ISP device into bridge mode.

Regards,
S.

OK, that is really interesting, thanks. I must read up on PPPoE and will give this another go tomorrow.

Really appreciate you taking the time to write such a detailed response.

[OPNonsense]

Just a small hint:  Fritzboxes do NOT have the ability to bridge their modems - the mode setting for the FritzBox as an IP-Client disables the FW & NAT and allows the Fritzbox to be a simple IP client on an existing network (meaning using something else as gateway & firewall)

Here's a reference in English:
https://en.easybell.de/help/telephone-configuration/fritzbox-telephony/using-a-fritzbox-as-ip-client/

If you have a DSL line from your provider - best solution is to invest in a simple DSL Modem like Draytek Vigor130 or Vigor167 - as alternative, you can use a device like the Deutsche Telekom's Speedport Smart 3/4 devices which DO allow bridged mode (modem only)

Then you can use the OPN to get to the internet 

Hope this is helpful . . .

Cheers,
R.Fox

PS:  If you are in Germany and understand German - Here's a good video which shows how to use the OPNSense behind a Fritzbox using exposed host:  https://www.youtube.com/watch?v=-7G6MSVmdn8

Hey, thanks for this!

I am actually in the UK using Zen Internet (they supply FRITZ!Box routers too), but as fate would have it, I am also German, and I was actually already aware of the video. Although I think that guy did some things differently to me and had a bit more of a complicated set-up. I will give it another watch, though.

[OPNonsense]

As far as I know - this only applies to the Cable modem Fritzboxes (6000 series) and not the DSL ones 

Cheers,
R.Fox

That feels like a ripoff..... Thanks for sharing this. I personally avoid FrtizBoxes or per say any OEM and Telco provided device, as you never know what catches it may bring.


Anyway If this is the case for our OP of this thread, if really he is not able to put the BOX into Bride mode only way how to have this worked out with his current setup is to have the FrtizBox in Routed mode, OPN WAN set on Static IP from the LAN Pool FrtizBOX provides and disable on OPN NAT to avoid Double NAT situation. Additionally I would disable FW on the FrtizBox.



The best option would be really as you said "Get a normal xDSL modem and put it into bridge mode".

My personal opinion is > network device on which you dont have control should not be in your home network.

Regards,
S.

Think I agree with that opinion!

Any suggestions regarding modem? Not necessarily a specific model, but what things to go for/avoid in general terms?

[rfox]

Think I agree with that opinion!

Any suggestions regarding modem? Not necessarily a specific model, but what things to go for/avoid in general terms?

Not sure how fast your DSL is - or whether it's Supervectoring or not - but. I'd start with going to Amazon UK site and type in "DSL Modem" and see what pops up - I know the Draytek Vigors seem quite popular -

Cheers,
R.Fox