OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • Profile of wrobelda »
  • Show Posts »
  • Topics
  • Profile Info
    • Summary
    • Show Stats
    • Show Posts...
      • Messages
      • Topics
      • Attachments

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

  • Messages
  • Topics
  • Attachments

Topics - wrobelda

Pages: [1]
1
General Discussion / OPN in a Proxmox VM, trying to spoof VM NIC to transparently relay to host NIC
« on: August 15, 2022, 06:36:07 pm »
(FYI, this is a cross-post from Proxmox forums, where I got no response: https://forum.proxmox.com/threads/trying-to-spoof-guest-nic-to-transparently-relay-to-host-nic.113600/#post-490741)

I am trying to set up OPNSense inside Proxmox. Unfortunately I cannot pass the interface as is, due to lack of IOMMU support, as I am running this in a nested Azure VM, so I need to get by using bridges.

The host configuration is:
– eth0
– vmbr0 with eth0 assigned to it

Code: [Select]
iface eth0 inet manual

auto vmbr0
iface vmbr0 inet manual
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0

The guest configuration is:
– VirtIO NIC attached to vmbr0, with MAC overridden using same address as the eth0
– Firewall: NO
– MAC Filter: NO

Running dhclient on eth0 or vmbr0 correctly discovers and assigns an IP address.

Now, I am trying to get the OPNSense in a VM to get that IP address instead and to relay its traffic via the vmbr0 transparently outside of the host. I have done something very similar previously between OpenWRT running in a VM and another VM, using OpenWRT's "trivial relay" (kmod-trelay, see https://forum.openwrt.org/t/howto-kmod-trelay/49610/2, also https://github.com/openwrt/openwrt/commit/c3bba7f8c61ee98265bcffef8ee86e22aa89bbe9), and despite that this particular case is much simpler, I can't get the VM to communicate with the ISP properly. I tried simply by spoofing the eth0's MAC address by setting the OPNSense VM's interface to it, but that's not enough.

I also checked the traffic on both ends using tcpdump, and, interestingly, vmbr0 does see the DHCP requests coming from the VM, and the ISP does respond, but that response never reaches the VM, nor the tap interface corresponding to the VM that Proxmox assigned to the bridge.

What am I missing here?

2
Virtual private networks / Routing Virtual IP traffic over VPN
« on: March 16, 2022, 11:33:58 pm »
I managed to set up selective routing via WG VPN for LAN clients (https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html). Now I also want to route traffic originating from a service bound to a Virtual IP (10.0.0.91) configured on the LAN interface, i.e. from the firewall appliance itself.

I can see that the service running is successfully bound to the Virtual IP, looking at the tcpdump -ni vtnet0 output:

Code: [Select]
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
23:31:34.414408 ARP, Request who-has 10.0.0.91 tell 10.0.0.100, length 46
23:31:34.414456 ARP, Reply 10.0.0.91 is-at 62:15:db:f6:60:1b, length 28
23:31:34.418506 IP 10.0.0.100.51915 > 10.0.0.91.80: Flags [S], seq 120292338, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1261429721 ecr 0,sackOK,eol], length 0


However, my understanding is that these packets internally do not pass the LAN interface table, so the selective routing rule I have set up for LAN hosts won't do. I tried searching for a similar issue solved here, but no luck.

Any idea how to proceed here? I will appreciate help.

EDIT: Seems related: https://forum.opnsense.org/index.php?topic=16252.0

EDIT: OK, seems this is the culprit: https://forum.opnsense.org/index.php?topic=23399.msg111294#msg111294

However, disabling the forced gateway does *not* create a set of manual rules here. Investigating further.

EDIT: OK, I re-added the manual floating rule to "let out anything from firewall host itself (force gw)", so that I can add another rule above it, except nothing I do has any effect. I am clueless.

3
22.1 Legacy Series / OPNSense on Azure messing up routes?
« on: March 10, 2022, 12:40:43 am »
I migrated to OPNSense from pfSense on my Azure (self-provisioned) appliance and everything is mostly fine, except one issue I have with the routes it it is setting up using DHCP, which render the 168.63.129.16 (https://docs.microsoft.com/en-us/azure/virtual-network/what-is-ip-address-168-63-129-16) unusable.

Have a look at the dump first:

Code: [Select]
root@gw:~ # netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.32.0.1         UGS         hn0
10.0.0.0/22        87.99.46.47        US          hn0
10.1.0.0/23        link#6             U           hn1
10.1.0.4           link#6             UHS         lo0
127.0.0.1          link#1             UH          lo0
168.63.129.16      link#6             UHS         hn1
169.254.169.254    172.32.0.1         UGHS        hn0
172.32.0.0/24      link#5             U           hn0
172.32.0.7         link#5             UHS         lo0

root@gw:~ # ifconfig
hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: WAN
options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
ether 00:22:48:4f:b5:c7
inet 172.32.0.7 netmask 0xffffff00 broadcast 172.32.0.255
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
hn1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
ether 00:22:48:4f:bd:3f
inet 10.1.0.4 netmask 0xfffffe00 broadcast 10.1.1.255
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Now, compare it to the output from the old pfSense appliance:

Code: [Select]
root@gw:~ # netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.32.0.1         UGS         hn0
10.1.0.0/23        link#6             U           hn1
10.1.0.14          link#6             UHS         lo0
127.0.0.1          link#2             UH          lo0
168.63.129.16      00:0d:3a:7d:83:47  UHS         hn0
168.63.129.16/32   172.32.0.1         UGS         hn0
169.254.169.254/32 172.32.0.1         UGS         hn0
172.32.0.0/24      link#5             U           hn0
172.32.0.4         link#5             UHS         lo0



root@gw:~ # ifconfig
hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=48001b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,LINKSTATE,TXCSUM_IPV6>
ether 00:0d:3a:7d:83:47
inet 172.32.0.4 netmask 0xffffff00 broadcast 172.32.0.255
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
hn1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=48001b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,LINKSTATE,TXCSUM_IPV6>
ether 00:0d:3a:01:7f:5f
inet 10.1.0.14 netmask 0xfffffe00 broadcast 10.1.1.255
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>


Notice that the pfSense routes both 168.63.129.16  and 168.63.129.16/32 using the WAN interface, whereas OPNSense somehow assigns 168.63.129.16 to LAN. Removing the latter rule manually restores the communication back.

What's happening here? Any ideas?

This is fairly crucial, since 168.63.129.16 handles communication with the Agent, including backing up.


4
Virtual private networks / Traffic routed arbitrarily over the Wireguad Interface despite disabled WG gw
« on: February 23, 2022, 05:37:59 pm »
(Note that this was originally reported on GitHub, as I suspect this is a bug: https://github.com/opnsense/core/issues/5592)

Describe the bug

I am migrating my setup from pfSense to OPNSense. Everything was OK so far, until Wireguard client VPN migration. I copied my config 1:1 from pfSense, which was a basic "client" connection to a remote VPN provider, accompanied by a selective traffic redirection for one of the LAN hosts. Used this guide and it worked from scratch: https://docs.netgate.com/pfsense/en/latest/recipes/wireguard-client.html

Now, with OPNSense, here's what's happening:

1. With my LAN host redirecting rule enabled, the host is not getting connected to the Web. Checked `wg0` interface on the firewall and seeing monitoring ICMP packets only.
2. After a reboot, somehow *all*  the WAN traffic is routed via `wg0`, although LAN hosts don't get the Internet (probably because of NAT is somehow messed up)
3. Since all WAN traffic is being routed, I naturally assumed that the WG gw must have taken over the WAN one. However:
- WG gw has a lower priority (255) vs the WAN gateway (254)
- WG gw is not marked as upstream, but WAN gw is
4. Still, I disabled the WG gw altogether, yet the traffic *still* shows on the wg0 interface (!), and the LAN hosts are still not connecting to the Internet
5. So I rebooted, and it's still the same (!!!). Gateway is down, all custom firewall rules disabled, yet the WAN traffic still shows on `wg0` :o
6. Only after disabling wg0 interface things actually got back to normal. Rebooting and re-enabling the interface doesn't bork it up, which is a clear indicator that enabling/disabling things (gateways?) is currently not deterministic.

FYI, I can reproduce this each time.

Expected behavior

– Should be able to selectively route the traffic over the Wireguard gateway.
– Gateway should respect the priority and upstream denotation.
– Disabling a gateway should revert back to the next one in priority
– Disabling a gateway should clean up (revert) all changes done to networking configuration

Environment
OPNsense 22.1.1_3-amd64


Pages: [1]
OPNsense is an OSS project © Deciso B.V. 2015 - 2023 All rights reserved
  • SMF 2.0.19 | SMF © 2021, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2