OPNsense on Proxmox

[spetrillo]

Hello all,

I am running my OPNsense firewall as a Proxmox VM. All my internal VLANs are Linux bridge vlans, with my WAN interface being a passthrough device. I am trying to get access to the OPNsense LAN, which is one of three vlans on one physical interface. I have connected my USB to ethernet dongle directly to the NIC with these three vlans and set the vlan on the dongle to vlan 1, which is the OPNsense LAN interface. I cannot ping nor can I gain access to the GUI. Am I doing something wrong? What am I missing?

Thanks,
Steve

[OPNenthu]

Just a thought, is the Proxmox firewall blocking?

[EricPerl]

It's not really clear where the VLAN management is done (proxmox or OPN).
Did you pass the entire bridge to the VM and do all the VLAN management there? That's what I do.

It wouldn't hurt if you shared a screenshot of the interface assignment screen.
VLAN 1 is parented to a vtnet bridge?
Is the vtnet bridge assigned as well?
I wonder what the native network is...

FWIW, I've ran into issues with tagging VLAN 1.
That ID is treated as default/native in various pieces of network equipment.

[Fkhan6601]

I have a similar setup. Here is what is would suggest:

Use two physical lan interfaces for opnsense and another for proxmox. Using the same interface, without firewall rules on proxmox, will allow users on lan to access your vm admin page.

If you dont have physical ethernet ports, you can use 2 dongles. Connect the first dongle and install promox. Set the first dongle as the promox interface with a unique address on a cidr you dont plan on using (like 192.168.200.200 on 192.168.200.1/24). Once you go through the setup for proxmox power off the computer, connect the second dongle and power back on. Now setup opnsense as a vm and assign only the second dongle to opnsense. Use proxmox to connect to the vm and go through the opnsense setup for the second dongle. If you cant see the new interface in opensense, you have a compatibility issue and will need to change the interface type in proxmox. The third dongle/interface will connect to wan. Once you configure opnsense, you can connect a laptop to the second adapter and navigate to the ui by using the ip address you set in opnsense (windows requires you to set the ethernet subnet). You should be good to go.

I dont know how many physical interfaces you have, so I am assuming you only have 1 ethernet interface.

You can use a single dongle, but that would require firewall setup on promox. Good luck.

[meyergru]

As I understand it, you have three VLAN bridges on the USB-to-Ethernet adapter. Thus, you break out the VLANs on Proxmox, so you should have three vtnet interfaces in OpnSense, one for each VLAN. This way, there would be no need to tag VLAN 1 in OpnSense, just use the untagged vtnetX.

It is probably easier to use just one VLAN-aware bridge in Proxmox and have OpnSense tag the VLANs itself. In that scenario, you would be right to tag VLAN 1 there. If VLAN 1 was also your main VLAN and Proxmox is using the same bridge, you would have to configure it to use VLAN 1, as well.

How to use a tagged VLAN is not very well documented in Proxmox, it works like this:

Code Select Expand

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
#Physical interface

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eth0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 1-4094
        bridge-mcsnoop 0
#Network bridge

auto lan0
iface lan0 inet static
        address 192.168.1.5/24
        gateway 192.168.1.1
        vlan-id 1
        vlan-raw-device vmbr0
        accept_ra 2
        autoconf 1
        pre-up echo 2 > /proc/sys/net/ipv6/conf/$IFACE/accept_ra
        post-up echo 2 > /proc/sys/net/ipv6/conf/$IFACE/accept_ra
#Proxmox LAN interface

source /etc/network/interfaces.d/*

As you can see, like with VMs or LXCs, each interface is connected to the VLAN-aware bridge with its respective tag. Thus, other than with non-VLAN-aware bridges, the IP assignment of Proxmox itself is not on the bridge interface, but on a virtual interface that Proxmox is connected to - namely the lan0 interface in the example above.

This way, you can assign any Proxmox client to any VLAN and have Proxmox itself on VLAN 1. OpnSense can then have either a single untagged interface and tag by itself or you can create multiple tagged vtnet interfaces for each VLAN, as you like.

[dseven]

My interpretation of the original post is that there is a client device - laptop or similar - with a USB Ethernet adapter, which is directly connected to the VLANed NIC on the proxmox server. The OP claims that they have "set the vlan on the dongle to vlan 1". My suspicion would be with that claim (although there could be multiple issues, of course). VLANs on client devices (e.g. Windows PCs) can be "challenging". It's usually a lot simpler if the PC is connected to a port on a managed switch where the desired VLAN is "native" (untagged), and another switch port, with all appropriate VLANs tagged, is connected to the server.

[EricPerl]

@meyergru: in my experience, you don't need to make the Proxmox bridge vlan aware if you pass it to OPN and do all the vlan management there.
But you have to if you intend to specify a VLAN ID for another VM on that same bridge. Otherwise, you just lock up the entire bridge (guess how I found out).

@dseven: I read it the same way (USB dongle on the client).
I've used VLAN directly on clients. On Windows, I had to get a PRO version of the Intel driver.
On Ubuntu, I had to tinker with the advanced network configuration (and likely relied on decent HW/driver).

The problem with such configs is that it's not that easy to troubleshoot, especially if you don't have an alternate way to access OPN.
In the OP's case, he might be able to look at fw logs or dump traffic at the console.
When the client is plugged in, the DHCP exchange should be visible.

With a managed switch, port mirroring is an option.

Then again, VLAN 1 is THE exception on my LAN wrt my convention to include the VLAN ID in the subnet.
I still have 1 in the subnet but I moved to 7 for VLAN ID because my switch configuration clearly was treating 1 in a special way.

[meyergru]

@meyergru: in my experience, you don't need to make the Proxmox bridge vlan aware if you pass it to OPN and do all the vlan management there.
But you have to if you intend to specify a VLAN ID for another VM on that same bridge. Otherwise, you just lock up the entire bridge (guess how I found out).

Yes, but that is exactly the reason why I usually do that: It is completely consistent for VMs and the physical connections alike. I thnik it is easier to understand and avoid mistakes like using a VLAN for both the bridge connecting interface and in the VM itself (which results in QinQ).

For normal VMs, I just have to device on which VLAN they reside. Only an OpnSense VM does have to tag by itself - which is consistent to the way a bare-metal OpnSense connected to a trunk port would have to be configured.