ATTENTION: This tutorial could potentially be outdated. I can't test this scenario anymore!
Because it took me multiple days of Troubleshooting, I want to share the solution with the community.
General Prerequisites:
1x Vodafone Business Cable in Germany (most likely the 500mbit/s or 1000mbit/s package)
1x Fritzbox 6591 from Provider
1x Static IP or IP Subnet from Provider
1x Opnsense
Short Explanation why its a pain to do:
The Fritzbox 6591 doesnt run in a Bridge Mode like the previous Fritzbox 6430.
It maps requests from internal devices (such as the Opnsense Firewall) - that have an IP from the external Subnet that the Provider has provided - to a MAC Adress of the Interface the request came from (Arp Table).
From that point on, all sessions initiated from the Opnsense into the Internet work, but not sessions initiated from the Internet to the Opnsense.
Getting it to work with a single static IP is easy and requires a single exposed Host in the Fritzbox, but once you want to use a whole Subnet in the Opnsense, things get complicated, because the Fritzbox wants a unique MAC for each Exposed Host.
Also, the configuration page of the Fritzbox 6591 only works on LAN1.
Long Explanation, aka TUTORIAL:
Goals:
1. Whole external /29 Subnet provided by Fritzbox 6591 working on the Opnsense.
2. Configuration Interface of Fritzbox 6591 working from Internal LAN.
3. Sessions from WAN to LAN/DMZ and LAN/DMZ to WAN both work.
4. SNAT and DNAT
5. Firewalling
Example Networks for the Tutorial:
1x External /29 Subnet (109.1.1.128/29) - WAN
1x Internal /24 Subnet (172.16.1.0/24) - LAN
1x DMZ /24 Subnet (10.0.1.0/24) - DMZ
1x Management /24 Subnet (192.168.178.0/24) - Fritzbox Network CONF
Example Hardware for the Tutorial:
1x Fritzbox 6591
1x OPNsense VM in Hyper V, Hypervisor needs at least 4 NICs - Tutorial won't use VLANs.
[STEP 1 - Reset Fritzbox]
1.1. Reset the Fritzbox to default settings and let the provider provision it. There cant be any vacant entries in the ARP Table, because the Fritzbox hides them and things won't work!
[STEP 2 - Configuration of the Hyper-V Hypervisor]
2.1. Create Following Virtual Switches:
- WAN - external - NIC1
- LAN - external - NIC2
- DMZ - external - NIC3
- CONF - external - NIC4
[STEP 3 - Connect Network - Fritzbox to Hypervisor]
The Fritzbox has LAN1, LAN2, LAN3, LAN4.
The Hypervisor has NIC1, NIC2, NIC3, NIC4.
3.1 Connect LAN1 (Fritzbox Configuration Network) to NIC4 (Hypervisor Vswitch CONF)
3.2 Connect LAN2 (Fritzbox external WAN Subnet) to NIC1 (Hypervisor Vswitch WAN)
[STEP 4 - Create and Configure OPNSENSE]
4.1. Create Opnsense VM:
- 4vCPU
- 8GB RAM
- 120GB SSD Storage
- NIC1 - connected to Vswitch WAN
- NIC2 - connected to Vswitch WAN
- NIC3 - connected to Vswitch WAN
- NIC4 - connected to Vswitch WAN
- NIC5 - connected to Vswitch WAN
- NIC6 - connected to Vswitch LAN
- NIC7 - connected to Vswitch DMZ
- NIC8 - connected to Vswitch CONF
4.2. Open the Opnsense Configuration Webpage and change all the "Interfaces" to following descriptions and networks, because Hyper-V uses hn0,hn1... etc for virtual nics its adviced to use those as prefix:
As the Subnet 109.1.1.128/29 (Gateway 109.1.1.129, Net Adress 109.1.1.128, Broadcast 109.1.1.135) has 5 IP Adresses the OPNSENSE can use, we put them as NIC 1 to 5. We cant use Virtual IPs because the Fritzbox demands one unique MAC per IP.
- NIC1 (hn0) - Description: hn0_wan_cab130 - IPv4 address: 109.1.1.130/29
- NIC2 (hn1) - Description: hn1_wan_cab131 - IPv4 address: 109.1.1.131/32
- NIC3 (hn2) - Description: hn2_wan_cab132 - IPv4 address: 109.1.1.132/32
- NIC4 (hn3) - Description: hn3_wan_cab133 - IPv4 address: 109.1.1.133/32
- NIC5 (hn4) - Description: hn4_wan_cab134 - IPv4 address: 109.1.1.134/32
- NIC6 (hn5) - Description: hn5_lan - IPv4 address: 172.16.1.254/24
- NIC7 (hn6) - Description: hn6_dmz - IPv4 address: 10.0.1.254/24
- NIC8 (hn7) - Description: hn7_conf - IPv4 address: 192.168.178.254/24
4.3. Now we have to create an IPv4 Gateway in "System - Gateways - Single"
- Name: wan_cab_gwv4 - Interface: hn0_wan_cab130 - IP adress: 109.1.1.129 - Upstream Gateway: YES - Priority: 240
4.4. Next go back to "Interfaces" select "hn0_wan_cab130" and in it set the Gateway to wan_cab_gwv4
4.5. In "Interfaces" set on Disabled "hn1_wan_cab131", "hn2_wan_cab132", "hn3_wan_cab133", "hn4_wan_cab134"
[STEP 5 - Configure Exposed Hosts in Fritzbox]
5.1 We have to do the following loop for each single external IP Adress our OPNSENSE has on its hn0 to hn4 interfaces. Please be careful that when the configuration has ended, the configuration is the same as in STEP 4.2. MIND THE /29 and /30 SUBNETS OR IT WONT WORK!
5.2 For hn0_wan_cab130:
- Initiate some sort of session, like searching for updates in the opnsense.
- Open Fritzbox Control Panel on 192.168.178.1 by connecting LAN1 of the Fritzbox to a Laptop. (When you are done, connect it back to the Hypervisor)
- In "Home Network - Network" you should be able to see a device called "PC-109-1-1-130"
- Go to "Internet - Permit Access - Port Sharing" - "Add Device for Sharing" - Select the device "PC-109-1-1-130" and check "Open this device completely for internet sharing via IPv4 (exposed host)"
- Now the Opnsense can fully use the IP 109.1.1.130/29 on hn0_wan_cab130
5.3 - Repeat for interfaces hn1_wan_cab131 through hn4_wan_cab134, just increment by one on each turn until all interfaces are exposed hosts.
- Go into the OPNSENSE, set subnet mask of hn0_wan_cab130 from /29 to /32 and set the gateway on "default". Then disable the interface.
- Enable hn1_wan_cab131, set subnet mask to /32 and the gateway to "wan_cab_gwv4"
- Initiate some sort of session, like searching for updates in the opnsense.
- Open Fritzbox Control Panel on 192.168.178.1 by connecting LAN1 of the Fritzbox to a Laptop. (When you are done, connect it back to the Hypervisor)
- In "Home Network - Network" you should be able to see a device called "PC-109-1-1-131"
- Go to "Internet - Permit Access - Port Sharing" - "Add Device for Sharing" - Select the device "PC-109-1-1-131" and check "Open this device completely for internet sharing via IPv4 (exposed host)"
- Now the Opnsense can fully use the IP 109.1.1.131/32 on hn1_wan_cab131
5.4 - Enable all Interfaces and set the configuration like in 4.2, 4.3 and 4.4.
[STEP 6 - Enable Fritzbox Configuration Page from LAN]
6.1 - Open Fritzbox Control Panel on 192.168.178.1 by connecting LAN1 of the Fritzbox to a Laptop. (When you are done, connect it back to the Hypervisor)
6.2 - Enable Advanced View (Right Top Corner, the dots)
6.3 - Go to "Home Network - Network Settings - Static Routing Table - IPv4 Routes"
6.4 - Add Following Route: Network: 172.16.1.0 Subnet Mask: 255.255.255.0 Gateway 192.168.178.254
6.5 - Connect the Fritzbox to the Hypervisor as in Step 3.1
6.6 - Create a Firewall Rule in the Opnsense that allows Traffic "Interface: hn5_lan, Source: hn5_lan net, Destination: 192.168.178.1, Port: HTTPS"
Now you can open the Fritzbox Configuration from the LAN Subnet using https://192.168.178.1
[STEP 7 - DNAT with Hairpinning]
Example:
- Webserver Port 80 with an A Record of webserver.example.tld - 109.1.1.131 - Internal IP in DMZ 10.0.1.131
- Mailserver Port 25 with an A Record of mail.example.tld - 109.1.1.132 - Internal IP in DMZ 10.0.1.132
7.1 - Go to "Firewall - Aliases" - Create Host Aliases: "host_webserver_example_tld - 109.1.1.131/32", "host_mailserver_example_tld - 109.1.1.132/32"
7.2 - Go into the Opnsense and open "Firewall - NAT - Port Forward" - Add a new Rule
7.3 - NAT Rule 1 (For Webserver): "Interface: hn1_wan_cab131, Protocoll: TCP, Source Address: Any, Source Ports: Any, Destination: This Firewall, Destination Port Range: from HTTP to HTTP, Redirect target IP: host_webserver_example_tld, Description: rule_nat_webserver_cab131_80, NAT reflection: Enable"
7.3 - NAT Rule 2 (For Mailserver): "Interface: hn2_wan_cab132, Protocoll: TCP, Source Address: Any, Source Ports: Any, Destination: This Firewall, Destination Port Range: from SMTP to SMTP, Redirect target IP: host_mail_example_tld, Description: rule_nat_mail_cab132_25, NAT reflection: Enable"
Now connections from the Internet to the Mailserver and Webserver will succeed, as will connections from the LAN because of the NAT Reflection (Hairpinning).
[STEP 8 - SNAT]
Now the DNAT for the Webserver and Mailserver work, but they both answer with IP 109.1.1.130 instead of their assigned .131 and 132. Manual SNAT Rules fix that.
8.1 Go into the Opnsense and open "Firewall - NAT - Outbound". Set the Mode to Hybrid outbound NAT rule generation. Save.
8.2 NAT Rule 1 (For Webserver): "Interface: hn0_wan_cab130, Source address: host_webserver_example_tld, Translation/target: hn1_wan_cab131 address"
8.3 NAT Rule 2 (For Mailserver): "Interface: hn0_wan_cab130, Source address: host_webserver_example_tld, Translation/target: hn2_wan_cab132 address"
8.4 NAT Rule 3 (Masquerading For LAN): "Interface: hn0_wan_cab130, Source address: hn5_lan Net, Translation/target: hn0_wan_cab130 address"
8.5 Set Mode to Manual outbound NAT rule generation and Save.
Now all outbound connections will have the right IPs. LAN net 172.16.1.254/24 uses 109.1.1.130/29, DMZ host_webserver_example_tld 10.0.1.131 uses 109.1.1.131 and so on.
[END]
This is the End of the Tutorial, don't forget to Check Firewall Rules for Outbound Traffic. For example for Internet Access from LAN.
Hope it helped someone. If there are any questions please post them, I will answer them.
Cheers!
Hi,
I got an OPNsense appliance with 8 NIC (Landitec Scope7 4220-xl). How do I get the four additional VNICs on one of my real NICs to connect to my FRITZ!Box 6591 Cable with a /29 subnet?
I'm new to OPNsense and don't know how to configure my interfaces correctly.
Thanks for your helping!
Quote from: Artist on April 17, 2022, 04:32:45 PM
I got an OPNsense appliance with 8 NIC (Landitec Scope7 4220-xl). How do I get the four additional VNICs on one of my real NICs...
Sorry for not answering earlier. The answer is by connecting 5 real NICs from your hardware Opnsense appliance to a Hardware Switch, and connecting that Switch to your fritzbox.
That way it still looks like 5 unique MAC addresses to the Fritzbox.
Mind that I cant test this tutorial anymore since I dont have the Vodafone Business Fritzbox anymore. If anybody uses this kind of setup I would love to hear.