Messages

In a test environment I was able to make this work. Having Layer 2 through a wireguard tunnel using VxLan over Wireguard. This was a fun experiment. Some one was trying to connect two lan with the same subnet and have the two distant network share the subnet and reach either side without apparent routing. The goal was to be able to move VM from one site to the other without having to setup the VM network config after the move. The only down side was that the VM would still use the far side gateway after the move, but still working perfectly with added latency obviously.

Here is the great picture of the operation.
You need an interface to manage the vxlan and then that interface need to be bridged with the lan interface, so your lan will become that bridge in the end.

Code Select Expand

LAN = bridge0 (vtnel1_lan, vxlan1)
vtnet1_lan = LAN interface
vxlan1 = vxlan interface that make this work
WAN = The WAN you know... lol
wg_net = the wireguard tunnel

Here is the overview of the interfaces, this was done in a VM lab, so WAN IP is RFC1918 in this case.



The VxLan is setup as such, those IP are from the wireguard tunnel, the Wireguard Instance IP is in Source Address and the Wireguard Peer IP in Remote Address.



Firewall rules for reference









Wireguard config overview for reference

Hi everyone,

I have a working WireGuard road-warrior setup, where I've assigned a private IPv4 address (RFC 1918) to my peer. I've also assigned a GUA IPv6 address from one of the delegated subnets I receive from my ISP, which provides me with a static /56 prefix. Additionally, I've tested using ULA and then using NPTv6 to translate that ULA to a GUA within one of the delegated subnets.

My question is: both solutions seem solid and secure, so why do many online guides recommend using ULA instead of GUA for this type of setup?

Best regards,
FrAllard

Hi,

I'm using ddclient to update the free and accountless https://myaddr.tools dynamic entry, they require the IP to be updated at least once every 90 days or else they delete that stale entry from their record.

I tried to find the information but I could not find it in the official documentation, does ddclient force update at regular interval even if IP of WAN doesn't change? If yes how often? If no, how can I do that? Cron to restart the service at interval?

For reference, if someone want to use that service too I use the native backend (to have http get) and created a Custom GET with the following URL in Server :

Code Select Expand

https://ipv4.myaddr.tools/update?key=<private_key_myaddr.tools_provide>&ip=__MYIP__
Since the creation of the DynamicDNS require a username and password I just put anything since it's irrelevant for that service. In hostname I entered the hostname I choose when creating the privatekey, again irrelevant for the update itself, since the private key takes care of that, but it's required by ddclient.

Nevermind I solved my problem... It was the vmware NSX-V router screwwing thing up ...

I changed the default admin distance of the gateway to 19 then made a static route for the other side subnet in the nsx-v with a admin distance of 1 and deleted the static route in my vm and it just worked.

I knew OPNsense was doing its job!

Doing traceroute from 10.65.87.21
traceroute 10.65.89.10 (udp) doesn't work... same behavior as above
traceroute -I 10.65.89.10 (icmp) works

Here is my problem at the moment.

I have a wireguard tunnel between two places, the tunnel works to an extend... Pings can flow through in both directions, tcp though can only flow in one direction and I really don't understand what is happening.

My OPNsense doesn't have a WAN interface, it's a wireguard gateway with a LAN interface and a wireguard interface. OPNsense has a gateway and that gateway is also the gateway of the hosts behind the problematic side of the tunnel.

So I have
10.65.87.1 (default gateway)
10.65.87.2 (OPNsense LAN)
10.65.87.21 (host that is trying to establish a tcp connection to the other side of the tunnel)
Routes of that host

Code Select Expand

$ ip r
default via 10.65.87.1 dev ens160 proto static
10.65.87.0/24 dev ens160 proto kernel scope link src 10.65.87.21
10.65.89.0/24 via 10.65.87.2 dev ens160 proto static
10.65.90.0/24 via 10.65.87.2 dev ens160 proto static
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br_pspdfkit_net proto kernel scope link src 172.18.0.1

Firewall rules on both side of the tunnel are wide open for now
Allow IPv4 * Source * Destination * everything *

I'm doing the following command on the 10.65.87.21 host : curl https:// 10.65.89.10:8006
(Inserted a space here to avoid the forum from creating a hyperlink)

Here is the capture done on the LAN interface on OPNsense, so packets do traverse the tunnel and get into the lan interface.

Code Select Expand

root@OPNsense:~ # tcpdump -n -i vmx0 host 10.65.89.10
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vmx0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:03:54.137159 IP 10.65.87.21.47858 > 10.65.89.10.8006: Flags [S], seq 3997485865, win 64240, options [mss 1460,sackOK,TS val 783913448 ecr 0,nop,wscale 7], length 0
15:03:54.144690 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340330385 ecr 783913448,nop,wscale 7], length 0
15:03:55.145738 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340331386 ecr 783913448,nop,wscale 7], length 0
15:03:55.146560 IP 10.65.87.21.47858 > 10.65.89.10.8006: Flags [S], seq 3997485865, win 64240, options [mss 1460,sackOK,TS val 783914458 ecr 0,nop,wscale 7], length 0
15:03:55.154193 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340331394 ecr 783913448,nop,wscale 7], length 0
15:03:57.161727 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340333402 ecr 783913448,nop,wscale 7], length 0
15:03:57.162497 IP 10.65.87.21.47858 > 10.65.89.10.8006: Flags [S], seq 3997485865, win 64240, options [mss 1460,sackOK,TS val 783916474 ecr 0,nop,wscale 7], length 0
15:03:57.169315 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340333410 ecr 783913448,nop,wscale 7], length 0
15:04:01.193744 IP 10.65.89.10.8006 > 10.65.87.21.47858: Flags [S.], seq 3568795638, ack 3997485866, win 65160, options [mss 1460,sackOK,TS val 1340337434 ecr 783913448,nop,wscale 7], length 0

Here is the tcpdump on  10.65.87.21 host trying to reach the other side, it never get the packets back even though opnsense does put it on the lan.

Code Select Expand

$ sudo tcpdump -n -i ens160 host 10.65.89.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens160, link-type EN10MB (Ethernet), capture size 262144 bytes
10:34:44.540703 IP 10.65.87.21.32778 > 10.65.89.10.8006: Flags [S], seq 4094067348, win 64240, options [mss 1460,sackOK,TS val 782163854 ecr 0,nop,wscale 7], length 0
10:34:45.544710 IP 10.65.87.21.32778 > 10.65.89.10.8006: Flags [S], seq 4094067348, win 64240, options [mss 1460,sackOK,TS val 782164858 ecr 0,nop,wscale 7], length 0
10:36:47.315603 IP 10.65.87.21.39506 > 10.65.89.10.8006: Flags [S], seq 2886622905, win 64240, options [mss 1460,sackOK,TS val 782286628 ecr 0,nop,wscale 7], length 0
10:36:48.328706 IP 10.65.87.21.39506 > 10.65.89.10.8006: Flags [S], seq 2886622905, win 64240, options [mss 1460,sackOK,TS val 782287642 ecr 0,nop,wscale 7], length 0
10:37:35.421371 IP 10.65.87.21.35480 > 10.65.89.10.8006: Flags [S], seq 1520415211, win 64240, options [mss 1460,sackOK,TS val 782334734 ecr 0,nop,wscale 7], length 0
10:37:36.424713 IP 10.65.87.21.35480 > 10.65.89.10.8006: Flags [S], seq 1520415211, win 64240, options [mss 1460,sackOK,TS val 782335738 ecr 0,nop,wscale 7], length 0
10:38:16.677116 IP 10.65.87.21.35994 > 10.65.89.10.8006: Flags [S], seq 4124073389, win 64240, options [mss 1460,sackOK,TS val 782375990 ecr 0,nop,wscale 7], length 0
10:38:17.704686 IP 10.65.87.21.35994 > 10.65.89.10.8006: Flags [S], seq 4124073389, win 64240, options [mss 1460,sackOK,TS val 782377018 ecr 0,nop,wscale 7], length 0

Hi!

Here is what I want to do. I have a WireGuard tunnel established to a VPS and I can ping the VPS from my LAN and also the VLAN I want to route through the tunnel. The ping is going through both ways without NAT pure routing.

I would like to force a VLAN to go through the tunnel to access Internet but I'm missing one last part I believe. Here is how it is setup at the moment.

WG IPs: 10.2.3.1 (VPS), 10.2.3.2 (OPNsense client)
LAN IPs: 10.20.30.1/24
VLAN IPs: 10.200.300.1/24

On the VPS WireGuard AllowedIPs include both LAN and VLAN subnets.
On the OPNsense side AllowedIPs is only 10.2.3.0/24

I created a Gateway like that
  Interface: WG Interface
  Gateway: 10.2.3.1

Then I create a Firewall Rule in the VLAN section to allow all and then set the gateway to the new one I created.

I did not create a NAT rule as I don't think I need one, the VPS is going to do nat, I want pure routing if possible between my devices on the VLAN all the way to the VPS then when it need to exit to Internet the VPS is going to do the natting.

I traced packets on the VLAN interface and the WG Interface and I can see the packets entering those interfaces. When I trace on the VPS the WG interface I can see the packets if the client on the VLAN is pinging the VPS WG IP, but if the client tries to ping let's say 1.1.1.1 I don't see any packets. Like if WireGuard does not route the trafic trough.

Resuming what I can see on the trace.
VLAN Client 10.200.300.128 ping 1.1.1.1
On the VLAN interface on OPNsense I see :

• 10.200.300.128 > 1.1.1.1 ICMP echo request
• no reply

On the WG interface on OPNsense I see :

• 10.200.300.128 > 1.1.1.1 ICMP echo request
• no reply

On the WG interface on the VPS I see :

• nothing

VLAN Client 10.200.300.128 ping 10.2.3.1 (The VPS WG IP Address)
On the VLAN interface on OPNsense I see :

• 10.200.300.128 > 10.2.3.1 ICMP echo request
• 10.2.3.1 > 10.200.300.128 ICMP echo reply

On the WG interface on OPNsense I see :

• 10.200.300.128 > 10.2.3.1 ICMP echo request
• 10.2.3.1 > 10.200.300.128 ICMP echo reply

On the WG interface on the VPS I see :

• 10.200.300.128 > 10.2.3.1 ICMP echo request
• 10.2.3.1 > 10.200.300.128 ICMP echo reply

[bb]

Hi,

I have a friend that need to install OPNsense behind his ISP device. The device need to be kept for phone and TV purpose. However he would like to use OPNsense for his own network. His provider is giving him GCNAT IPv4 + Native IPv6 to the ISP box. Once we connect OPNsense on a lan port of the ISP box the WAN receive a DHCPv6 address in the same /64 as the isp box. In fact the WAN address is being given a /128 I don't know why. The ISP box status seems to say that it has received a /56. I can confirm that trafic for every /64 under the /56 is being routed to the isp box.

Enough of the backstory! Here is the problem. We tried WAN DHCPv6 and ask for a /64 delegation and LAN Track WAN Interface. It's a no go, LAN doesn't get an IPv6.

We tried to enable DHCPv6 Relay on the LAN by giving one of the /64 address from the /56. I packet capture the WAN while forcing a DHCPv6 client on the LAN to request an address and the relay packets are being sent to the box, but no answer...

Which IP addresses should we setup for the WAN and the LAN and eventially the LAB and enable DHCPv6 Relay and have it work?

I'm also wondering if we'll even be able to have a separate LAN and LAB because I'm wondering how are we gonna tel the isp box that it need to route the trafic of that /64 to the WAN on OPNsense?

Here are an example of what the IPv6 addresses look like at the moment.
ISP Box status tell us we have a delegation looking like that : 2001:1234:5678:99::/56
ISP Box LAN address: 2001:1234:5678:9901::1/64
ISP Box deliver DHCPv6 in that range.

We tried giving the OPNsense LAN IPv6 : 2001:1234:5678:99bb::1/64
Then enable DHCPv6 Relay on the LAN interface. The capture on the WAN was showing all the right address and was showing relaying to 2001:1234:5678:9901::1 as instructed.

We looked everywhere at example on how to successfully setup DHCPv6 Relay on LAN.

Can anyone please help?

Dans la configuration de ton tunnel IPSec tu dois ajouter une Phase 2 qui inclus tes adresses OpenVPN.
Tu dois nécessairement ajouter cette phase 2 des deux côtés du tunnel IPSec.

Si tu n'as pas accès à reconfigurer l'autre côté du tunnel il est peut-être possible de le faire avec de la NAT de Sortie (Firewall -> NAT -> Outbound) avec une règle manuelle, mais je ne l'ai jamais fait comme ça et c'est toujours mieux de router le trafic plutout que le natter.

Yeah I saw your post, since it was for a Chelsea, if I remember correctly, I didn't want to hijack your thread.

I have a QLogic NetXtreme II BCM57810 10GbE. It's one of those card that is "compatible" with the sfp my isp install in their equipment. So I took it out installed it in that card, modified the bios of the card a little to allow 2.5 GBps link and modified and recompiled the driver to allow the driver to link at 2.5 Gbps also.

If I could find a good Linux equivalent to OPNsense/pfSense I'd go with that probably. I've learned that pppoe under FreeBSD is single threaded. I have a TrueNAS also and it too is struggeling with 10 Gbps. I can only hit about 5 Gbps with it.

I really hope we can find a solution to this performance problem. I kind of really like OPNsense. I've spent too many years to be able to count with pfSense, but since they changed the way they handle their business I wanted to change for OPNsense, when I crashed my pfSense I saw the perfect opportunity to switch. Loosing more than 25% of performance is not an option though.

I have 1.5 Gbps internet. My NIC host the GPON from my provider and I have a modded driver that allow OPNsense to link at 2500 Mbps. All that is working, but my speed isn't on par with what I was easily getting with pfSense with the same exact hardware. I'm wondering which settings I need to adjust to get my speed back.

Is can do a speedtest that will show 1400 Mbps and the next one seconds after the first will show 1000 Mbps. The speed is very inconsistent. The speed also ramps up slowly, where with pfSense the speed was instantaneous.

When I download on Usenet, I was getting consistently 1.5 Gbps speed, but with OPNsense I cannot get faster than 1 Gbps.

I have no IDS and anything other than basic firewall rules enabled. I did try to create a trafic shaper with fq_codel, but I deleted eveything in that section since I was still having speed problem.

Here is the dmesg regarding that nic.

root@OPNsense:~ # dmesg | grep bxe0
bxe0: <QLogic NetXtreme II BCM57810 10GbE [BELL BYPASS] (B0) BXE v:1.7> mem 0xdd800000-0xddffffff,0xdd000000-0xdd7fffff,0xde010000-0xde01ffff irq 17 at device 0.0 on pci2
bxe0: PCI BAR0 [10] memory allocated: 0xdd800000-0xddffffff (8388608) -> 0xfffff800dd800000
bxe0: PCI BAR2 [18] memory allocated: 0xdd000000-0xdd7fffff (8388608) -> 0xfffff800dd000000
bxe0: PCI BAR4 [20] memory allocated: 0xde010000-0xde01ffff (65536) -> 0xfffff800de010000
bxe0: Found 10Gb Fiber media.
bxe0: IFMEDIA flags : 20
bxe0: Using defaults for TSO: 65518/35/2048
bxe0: Ethernet address: 00:0e:1e:82:cd:d0
bxe0: MSI-X vectors Requested 5 and Allocated 5
vlan0: changing name to 'bxe0_vlan35'
bxe0:
bxe0: link state changed to UP
bxe0_vlan35: link state changed to UP
bxe0: link state changed to DOWN
bxe0_vlan35: link state changed to DOWN
bxe0: ERROR: Changing VLAN_HWTAGGING is not supported!
bxe0: ERROR: Changing VLAN_HWFILTER is not supported!
bxe0: NIC Link is Up, 2500 Mbps full duplex, Flow control: ON - receive & transmit
bxe0: link state changed to UP
bxe0_vlan35: link state changed to UP
bxe0: ERROR: Changing VLAN_HWCSUM is not supported!
bxe0: link state changed to DOWN
bxe0_vlan35: link state changed to DOWN
bxe0: NIC Link is Up, 2500 Mbps full duplex, Flow control: ON - receive & transmit
bxe0: link state changed to UP
bxe0_vlan35: link state changed to UP
bxe0: ERROR: Changing VLAN_HWTAGGING is not supported!
bxe0: ERROR: Changing VLAN_HWFILTER is not supported!
bxe0: ERROR: Changing VLAN_HWCSUM is not supported!

The same nic has two ports avec un LAN port has a 10 Gbps connection to my 10 Gb switch and tested the speed between my server and the LAN port and I was only getting 1.5 Gbps. SOmething in the optimization of that nic is missing.

Please help!