Hello Guys,
this is my first topic on this forum. I use OPNsense for around 1.5 Years now and im happy with it.
Yesterday i switched Servers and had to resetup IPv6. Since then i cant get a route out via a Client in my LAN Network.
Clients in LAN can ping the LAN Adress of the OPNSense and also its WAN address, but nit the WAN GW or any other Target in the Internet. I also mentioned that the OPNSense cant ping the Internet or WAN GW from the LAN Address.
I tried a lot, including different Firewall Rules, MTU´s etc, but no success. So i try to post my problem here to get help from experts.
The Setup looks quite the same on my old OPNSense where this Problem didn´t happens. the only difference is that on the old one the WAN GW wars a llink-local address (fe80::1) so the WAN subnet is /128.
My Setup on the new OPNSense looks like this: (IPv6 Address is a bit randomized)
Public Subnet form Hetzner vSwitch: 2a1:4f8:f01a:1a69::/64
WAN GW: 2a1:4f8:f01a:1a69::1
WAN IP: 2a1:4f8:f01a:1a69::2/126
MTU WAN: 1500
LAN Interface Setup: 2a1:4f8:f01a:1a69:1::1/123
LAN IP: 2a1:4f8:f01a:1a69:1::1/123
MTU LAN: 1350
(MTU`s are correct IPv4 is working perfectly and every device knows the MTU`s)
It would be great to get it working. If so, i will write a guide for IPv6 on Hetzner vSwitch with OPNSense, because there isn't any.
Thanks for your help!
Quote from: niclas on November 14, 2024, 12:32:37 AM
WAN GW: 2a1:4f8:f01a:1a69::1
WAN IP: 2a1:4f8:f01a:1a69::1/126
That doesn't look right - you can't be your own gateway?
@dseven Oh sorry, I made a mistake in writing:
WAN IP is: 2a1:4f8:f01a:1a69::2/126
OK, so then is the 2a1:4f8:f01a:1a69:1::/123 prefix routed to 2a1:4f8:f01a:1a69::2 at the upstream gateaway (presumably something in the Hetzner vSwitch)? If not, you'd have to do some sort of NAT......
Yes Hetzners Gateway is 2a1:4f8:f01a:1a69::1 and the OPNSense need to route outgoing Traffic over it. But form LAN I can't ping it ore something outside. If I ping it via the OPNSense with WAN IP as Source the Ping success.
And if I tried to set the WAN IP as Upstream Gateway on the LAN interface of OPNSense it can ping outside via LAN IP, but a client can't. The clients even can't ping the OPNSense, before undo the LAN GW
Quote from: niclas on November 14, 2024, 02:24:39 PM
Yes Hetzners Gateway is 2a1:4f8:f01a:1a69::1 and the OPNSense need to route outgoing Traffic over it. But form LAN I can't ping it ore something outside. If I ping it via the OPNSense with WAN IP as Source the Ping success.
That doesn't answer the question - the upstream gateway will need to have a route for 2a1:4f8:f01a:1a69:1::/123 pointing to your OPNsense WAN interface at 2a1:4f8:f01a:1a69::2 - without that, responses won't find their way back to your OPNsense LAN.
I can't influence the Upstream GW, because it's managed by Hetzner. Dose this mean the OPNSense need a /64 Subnet on WAN?
In my Previous Subnet The Upstream wars a link-local Adress, why dose this didn't need a route?
If you want to have an IPv6 prefix behind by OPNsense, either that prefix needs to be routed to OPNsense's WAN interface, or you'd have to do NAT.
Presumably when you were using a LLA, you had a routable prefix that was routed to your LLA...?
So I have to use NAT now if I have a LAN and DMZ that i wanna give a IPv6?
Can I use VIP for IPv6 to give a separate IPv6 on WAN for Routing?
Previous the Gateway wars a link local on wan given by Hetzner. So my WAN look like this: 235:248:241::1 and the LAN and DMZ had something like this: 235:248:241::1:1/123. It worked out of the Box.
I'm new to IPv6 so it's a bit complicated to understand for me 😅
Quote from: niclas on November 14, 2024, 03:19:57 PM
So I have to use NAT now if I have a LAN and DMZ that i wanna give a IPv6?
Or you order one additional /64 for LAN and one additional /64 for DMZ and Hetzner will in their standard procedure route these to the WAN address of your OPNsense.
You cannot meaningfully subnet a /64 in IPv6. Each interface gets a /64 - always. Not larger, not smaller.
But if OPNSense don't claim the additional Subnet, how dose the Hetzner GW know that it has to route it to the OPNSense?
If I wanna use NAT and give my clients local IPv6 addresses how can i claim for example ...::4:10 as WAN destination for my client?
Quote from: niclas on November 14, 2024, 03:47:29 PM
But if OPNSense don't claim the additional Subnet, how dose the Hetzner GW know that it has to route it to the OPNSense?
You order the additional /64 from Hetzner and they route it statically to the single MAC address where they route everything else, too. Which is supposedly WAN of your OPNsense. Then you configure one address of that /64 statically on the DMZ interface - this becomes the default gateway for your VMs.
Ah now i understood the key difference. The Subnet I used before war's bound to a MAC-Address, but the new one not. On vSwitch you claim your IPv4 and v6, but there is no need for a specific MAC.
How can I bind a second IPv6 from the Subnet to the OPNSense?
As an alias address on WAN.
If you want an address on LAN or OPT1, then routing must take place and you must use an entire /64.
I don't know if Hetzner support routing additional /64 with a vSwitch. They sure do if you do not use a vSwitch, though.
There's also going to be this soon:
https://github.com/opnsense/plugins/pull/4348
With it I managed to put a /128 IPv6 address on the LAN interface thats in the same /64 as the WAN address and this proxy then proxies the NDP message to make it discoverable by the Provider Edge router.
Though configuration and testing is a little arcane, I need more information from people who actually use it for more than that for proper documentation.
Will it work the other way round? That would be our setup at Hetzner for hosting - not OPNsense but FreeBSD.
WAN: dead:beef:dead:beef::1/128
WAN GW: fe80::1%igc0 (for example)
LAN: dead:beef:dead:beef::2/64 (bridge for our hosting jails and their default GW)
Kind regards,
Patrick
I really do not know, the ways it works are all in the man pages. I did a lot of testing and I have read people who use it here:
https://gist.github.com/MCterra10/7e3930e54db0be10f42dd999e3263560?permalink_comment_id=5178523#gistcomment-5178523
I could not recreate the above mentioned setup yet.
https://man.freebsd.org/cgi/man.cgi?query=ndproxy&apropos=0&sektion=4&manpath=FreeBSD+11-current&format=html
I guess you have to test the potential of it yourself in your environment, but the module seems to be around for a long while and there are no reports of people who have issues with it. Any (good or bad) reports are scarse...
Quote from: Patrick M. Hausen on November 14, 2024, 04:41:18 PM
Will it work the other way round? That would be our setup at Hetzner for hosting - not OPNsense but FreeBSD.
WAN: dead:beef:dead:beef::1/128
WAN GW: fe80::1%igc0 (for example)
LAN: dead:beef:dead:beef::2/64 (bridge for our hosting jails and their default GW)
Kind regards,
Patrick
Yes i had it like this before, but that only works if Hetzner gives you a MAC that you have to use. As mentioned earlier, because of that they know where to route your subnet.
vSwitches are different. You claim a IP without giving your IP to Hetzner so the GW dosen't know where to route the rest of the /64 subnet if your WAN only claims 128. (If I understood it right)
Correct. But do you really need a vSwitch?
Yes, we use a Proxmox Cluster and if you wanna migrate your vm's / your OPNSense you need to change the ip's if you don't get the IP via vSwitch.
Hetzner only offer Public IP`s bound to a Dedicated Server or a vSwitch.
Quote from: Monviech (Cedrik) on November 14, 2024, 04:45:55 PM
I really do not know, the ways it works are all in the man pages. I did a lot of testing and I have read people who use it here:
https://gist.github.com/MCterra10/7e3930e54db0be10f42dd999e3263560?permalink_comment_id=5178523#gistcomment-5178523
I could not recreate the above mentioned setup yet.
https://man.freebsd.org/cgi/man.cgi?query=ndproxy&apropos=0&sektion=4&manpath=FreeBSD+11-current&format=html
I guess you have to test the potential of it yourself in your environment, but the module seems to be around for a long while and there are no reports of people who have issues with it. Any (good or bad) reports are scarse...
I will give it a try, if i see it correct its experimental at the moment. Will it be implemented as plugin later? I can try it in a test enviroment, but not at production at the moment.
Check the previous page, it will come as normal plugin in the next version. Any tests are highly valuable for documentation purposes. Thanks in advance.
Quote from: Patrick M. Hausen on November 14, 2024, 04:08:22 PM
As an alias address on WAN.
If you want an address on LAN or OPT1, then routing must take place and you must use an entire /64.
I don't know if Hetzner support routing additional /64 with a vSwitch. They sure do if you do not use a vSwitch, though.
Where do i set the aliases? I tried the Virtual IP as i use it on IPv4, but that dosen't work.
The plan is to set multiple WAN IPv6 Addresses and then do a Port-forwarding and Outbound NAT for the Servers.
Quote from: Monviech (Cedrik) on November 14, 2024, 08:58:29 PM
Check the previous page, it will come as normal plugin in the next version. Any tests are highly valuable for documentation purposes. Thanks in advance.
That sounds great! Sounds like it solves my Problem without NAT. I will test the Plugin if its out.
When it will come? In v25 or v24.8?
Are there any settings to do or dose it work out of the box?
Probably in the next minor version. :)
So 24.7.9
Theres 4 settings, check out the man page. Its only 4 settings but it feels rather complicated (at least to me) even though it should be simple. Guess it depends highly on the exact usecase.
Quote from: Monviech (Cedrik) on November 14, 2024, 09:02:27 PM
Probably in the next minor version. :)
So 24.7.9
Theres 4 settings, check out the man page. Its only 4 settings but it feels rather complicated (at least to me) even though it should be simple. Guess it depends highly on the exact usecase.
Ah, found it. Can I do multiple LAN Networks? Because i have to put the MAC and IP in the config.
Quote from: niclas on November 14, 2024, 08:58:38 PM
Where do i set the aliases? I tried the Virtual IP as i use it on IPv4, but that dosen't work.
Surprises me although I admit I never tried it. Yes, I implied Virtual IP. Should work with IPv4 and IPv6 just the same.
I never used it because that's not how IPv6 is supposed to work. NAT deserves to die.
Quote from: Patrick M. Hausen on November 14, 2024, 09:09:39 PM
Quote from: niclas on November 14, 2024, 08:58:38 PM
Where do i set the aliases? I tried the Virtual IP as i use it on IPv4, but that dosen't work.
Surprises me although I admit I never tried it. Yes, I implied Virtual IP. Should work with IPv4 and IPv6 just the same.
I never used it because that's not how IPv6 is supposed to work. NAT deserves to die.
😂 Thats what a wanted, but then the GW Thing stopped me. :(
I will try it. The VIP shoud be pingable if the FW Rule for it is set right? (Ipv4 is it)
The default GW in a vSwitch at Hetzner is not "pingable".
For IPv4 they told me to use 78.46.170.2 for gateway monitoring in Falkenstein. Best open a support ticket and ask which IPv6 address to use.
The Gateway for IPv6 on a vSwitch is pingable. :)
The Virtual IP form type "Alias IP" also works now (no idea why not last time), but it takes some time to become active...
Quote from: niclas on November 14, 2024, 09:08:36 PM
Quote from: Monviech (Cedrik) on November 14, 2024, 09:02:27 PM
Probably in the next minor version. :)
So 24.7.9
Theres 4 settings, check out the man page. Its only 4 settings but it feels rather complicated (at least to me) even though it should be simple. Guess it depends highly on the exact usecase.
Ah, found it. Can I do multiple LAN Networks? Because i have to put the MAC and IP in the config.
No it can only proxy for one network. Other vendors offer an enterprise level implementation of the same feature that support multi interfaces.
https://www.juniper.net/documentation/us/en/software/junos/neighbor-discovery/topics/topic-map/ndp-dad-proxy.html#concept_m34_4lq_qsb__section_xzt_g2p_ssb
Though this shows again how bad network design choices by ISP spawn new "features" to fix things that are just as bad as NAT. If everybody would adhere to IPv6 standards everybody would have properly routet subnets. IPv6 is dirt cheap, theres no reason to make it hard for everybody but greed and bad choices.
Quote from: Monviech (Cedrik) on November 14, 2024, 10:02:30 PM
Though this shows again how bad network design choices by ISP spawn new "features" to fix things that are just as bad as NAT. If everybody would adhere to IPv6 standards everybody would have properly routet subnets. IPv6 is dirt cheap, theres no reason to make it hard for everybody but greed and bad choices.
Word, bro! 8)
;D
Thanks for Your Help!!!
Now i understood the Problem and also have 2 Solutions for it.
I also wrote the Hetzner Support about it, lets see what they thing about it.
I will use NAT for now and then try the nd-proxy. After implementing it in production i will write a guide on Hetzner for it.
Thanks and i keep you updated on nd-proxy :)
Quote from: niclas on November 14, 2024, 10:42:46 PM
Thanks for Your Help!!!
Now i understood the Problem and also have 2 Solutions for it.
I also wrote the Hetzner Support about it, lets see what they thing about it.
I will use NAT for now and then try the nd-proxy. After implementing it in production i will write a guide on Hetzner for it.
Thanks and i keep you updated on nd-proxy :)
The answer from Hetzner is: "On vSwitch its not possible to set a MAC for your Subnet, but Yyou can add up to 32 Mac Adresses (Server) to a vSwitch, so NAT is not needed. "
But if you wanna use the Features from OPNSense like Geoblocking or the FireHole Lists etc. you need it.