OPNsense Forum

Hey guys,

First off, thanks for looking! I was wondering if this is even possible and would highly appreciate any input.

I have an Opnsense Virtual Machine on a server (running VMWare). Because I already have a hardware firewall, I would like to use Opnsense as nothing more than a VPN device, meaning I disabled all firewall rules and NAT on the VM (all of which are taken care of with the hardware firewall), and I'm only utilizing the em0 to LAN interface on the VM.

I created the appropriate NAT rules and set the appropriate rules on the hardware firewall, and after following the IPSEC roadwarrior VPN tutorial, I'm able to connect to the virtual machine and attain an ip address within the LAN for my iPhone.

The thing is, I cannot access anything else on the network other than what's in the LAN (VLAN 200).

(https://i.imgur.com/rv3mhLe.png)

I would like to be able to access VLAN 50 and VLAN 10 once I connect to the Opnsense VPN on VLAN 200. I should also point out that VLAN tagging occurs at the vSwitch level, and thus there is no need to set the VLAN tags within the Opnsense virtual machine.

I have tried setting up routes on the VM, and I see that pings (or other traffic) are able to get out and come back to the virtual machine, but once it hits the Opnsense virtual machine coming back, it terminates and stops the packet flow.

Is there anyone out there who has successfully set up such a configuration? If so, I'd really appreciate any input on doing the same? Is such a configuration even optional (meaning only using LAN on the em0 interface) and even so, are there more effective ways to do this kind of thing?

Thanks again for all your time!

-Dave

Hi Dave,

This is definitely possible. Sounds like your main firewall is also the router and I am guessing the OPNsense has a static IP setup... in that case the routing table needs to be extended to know the gateway and destination network for your other VLANs going over the main firewall.


Cheers,
Franco

And the "main firewall" needs to know the VPN networks for the route back.
I would say that the OPNsense VM knows all required routes as it has only two: The VPN network and the default gateway.

Thanks for your reply, guys!

I configured the appropriate gateways and all routes and still no luck.

I can get my phone to access the LAN (VLAN 200) and ping the Opnsense VM as well as the hardware firewall gateway, but still no access to any other VLANs or the external network. For what it's worth, the logs on the hosts in the other VLANs reveal that I am pinging the devices and packets are being sent back.

This to me says that packets are leaving my phone, hitting my hardware firewall, getting to the virtual machines in other VLANs, getting replies, going back all the way up until they come back to the Opnsense VM and then getting dropped.

Oddly enough, the Opnsense Virtual Machine itself is capable of accessing other VLANs, the external network AND can ping my phone with no issues. I've added another diagram below to show this.

(https://i.imgur.com/freSyL6.png)

Again, thank you all for your time.

-Dave

the next thing I would try is running tcpdump to see where the pings will go to.

May this be another of those "Firewall: Settings: Advanced: Disable Reply-To" cases?


Cheers
Franco

@franco: I would not expect that to change anything if the firewall is disabled.

QuoteThe thing is, I cannot access anything else on the network other than what's in the LAN (VLAN 200).

these are clients that are connected to Your HW Firewall?

Could You print Your routing table of Your opnSense device, please?

Cheers, Stephan

@fabian not necessarily. Disabling NAT and all rules does always mean disable pf (extra setting) so some automatic gateway rules may still be in the ruleset.

@fabian
Do you mean run it on the Opnsense VM itself?

@franco
I tried that as well. Made no difference

@Stephan
Here's my current routing table:
http://uploads.im/Nl24O.jpg

Btw, I wanted to thank you guys again. I truly appreciate your time.

-Dave

Quote from: davidm on September 22, 2017, 08:47:19 PM
@fabian
Do you mean run it on the Opnsense VM itself?

yes - via the GUI with the packet capture page or the CLI using SSH. Would recommend the CLI.

Quote from: fabian on September 22, 2017, 09:33:46 PM

Quote from: davidm on September 22, 2017, 08:47:19 PM
@fabian
Do you mean run it on the Opnsense VM itself?

yes - via the GUI with the packet capture page or the CLI using SSH. Would recommend the CLI.

Pinging from the iPhone VPN client to Google's DNS:
13:19:00.420513 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 1, length 64
13:19:01.065023 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:01.460413 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 2, length 64
13:19:01.864592 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:02.449406 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 3, length 64
13:19:02.763523 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:03.480193 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 4, length 64
13:19:03.487601 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:04.186267 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:04.480168 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 5, length 64
13:19:05.086951 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:05.490400 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 6, length 64
13:19:05.885036 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:06.490236 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 7, length 64
13:19:06.586857 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:19:07.490062 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 8, length 64
13:19:08.490251 IP 172.16.200.3 > google-public-dns-a.google.com: ICMP echo request, id 5317, seq 9, length 64
13:19:08.498396 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46


Pinging to the iPhone VPN client from my laptop on a different VLAN:
13:21:34.336308 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:35.035239 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:35.934876 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:36.734680 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:37.333973 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:38.346563 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:38.946088 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:39.845858 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:40.546805 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:41.146561 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:42.361434 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:42.961067 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:43.860648 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:44.661290 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46
13:21:45.561126 ARP, Request who-has 172.16.200.3 tell 172.16.200.1, length 46

This output looks quite strange as it seems to be only the output of traffic from the phone to the firewall but no traffic in the reverse direction.

For the second it seems that there is no data received by the firewall. Check the client firewall and routing table (especially if the network is known).

Update*

Getting one step closer.

After doing some more reading, I decided to enable proxy-arp on the virtual IP address of the LAN network (Firewall > Virtual IPs > Settings). I can now ping the cell phone from my router and I see the arp responses going to the appropriate IP. Still not able to ping out or the gateway though. Now back to more troubleshooting.  :-X

Quote from: davidm on September 27, 2017, 08:12:28 PM
Update*

Getting one step closer.

After doing some more reading, I decided to enable proxy-arp on the virtual IP address of the LAN network (Firewall > Virtual IPs > Settings). I can now ping the cell phone from my router and I see the arp responses going to the appropriate IP. Still not able to ping out or the gateway though. Now back to more troubleshooting.  :-X

Welp, I take that back... I rebooted the VM and now it's not working again. I'm thinking of biting the bullet and getting commercial support. Anyone have any experience? I was quoted 299 Euro for 3 hours of support. Would this suffice? Any tips on conveying my message more accurately as to not confuse the support or make them more efficient in troubleshooting this? Thanks!