Hello all,
I have seen some posts on running OPNsense as a vm, as well as some older guides on the Internet. Does anyone have an up to date guide on how to do this? I am trying to consolidate all my servers as vms, for ease of use and management. I have gen7 i5 with 24 gigs of RAM to build on.
Thanks,
Steve
Hi Steve,
There's not much to it; boot a FreeBSD 12 64-bit VM from the ISO and assign the vNIC's for the external networks or VLAN's. I use 2 vCPU and 4 GB of RAM which comfortably exceeds the system requirements.
Don't forget the os-vmware plugin.
Bart...
Bart,
Thanks for the info...maybe I am making more of it than I really have to. One question regarding the specs you have provisioned for the vm. Are you running any services, other than those that come with the default install?
Thanks,
Steve
Light use (e.g. OpenVPN server) VM uses a few 100 MB RAM and very little CPU.
More detail in PM I sent you.
Bart...
For those of you that have viirtualized OPNsense do you use the vnics or do you passthrough to the physical NICs? If you passthrough how does that affect the use of vnics that sit on those physical NICs for other vms? Do I need to have other NICs for the other vms?
Quote from: spetrillo on January 20, 2022, 10:19:21 PM
If you passthrough how does that affect the use of vnics that sit on those physical NICs for other vms? Do I need to have other NICs for the other vms?
If you pass through a NIC (or any other PCIe device) to a VM in ESXi, that NIC is not available for ESXi. It's exclusively inside that VM. I do that for NVME drives with TrueNAS SCALE in ESXi.
Quote from: spetrillo on January 20, 2022, 10:19:21 PM
For those of you that have viirtualized OPNsense do you use the vnics or do you passthrough to the physical NICs? If you passthrough how does that affect the use of vnics that sit on those physical NICs for other vms? Do I need to have other NICs for the other vms?
I've used OPNsense for years as a VM (on an ESXi server) and had no problems maxing out 1GB download speed on my last ISP where I had an FTTP connection, I used vNICs without any problems.
Should I use the VMXNET3 or E1000 adapter for any of my network connections? I see the SRV IO passthrough option but not going to use that just yet. I actually could use the SRV passthrough for the WAN interface, since the fw is the only device that will ever interface to the WAN.
I'd suggest you use SR-IOV as it's the most performant but if you don't want to then VMXNET3 and I'd ignore the E1000 unless you really can't use anything else.
Ok some more questions...
I have several vlans that I want to attach to my OPNsense vm. Not knowing if I am doing it right or wrong I created a VSS for each vlan, so I have a total of 5 virtual switches:
1) WAN
2) LAN/Mgmt
3) Wifi
4) Streaming
5) Server
So my OPNsense vm has 5 vnics attached, one for each vlan coming off of a separate virtual switch. When I boot the OPNsense vm in this config the LAN port takes on the IP of the streaming vlan, even without defining the streaming vlan to the vm. Not sure how this could happen, but I am assuming there is some form of cross contamination with the 5 vnics/vswitches?
there are three areas to configure: physical networks (the nics you have in your ESXi I assume), those you assign in the virtual switch config area to a nic, and the vlan configuration is happening in the port group area.
from your description I assume you missed or messed up the vlan config in the port group area, since you assign port groups to the nics in your OPNsense virtual machine.
Quote from: spetrillo on January 22, 2022, 01:17:22 AM
I created a VSS for each vlan
On a stand-alone ESXi host you generally create a virtual switch per physical switch. You define VLAN's across your entire site and trunk the VLAN's that connect to virtual machines to your ESXi.
To maximise aggregate throughput and availability, you can have more than one uplink between your vSwitch and your physical switches. A single link is fine for a home system where you'll likely have more single points of failure and won't generate enough traffic to saturate the link.
In your case, create one vSwitch and configure your external switch to tag the four VLAN's to the ESXi port (five if your WAN is a VLAN in the trunk). Configure port groups with the same VLAN on the vSwitch and test.
Bart...
Quote from: bartjsmit on January 22, 2022, 10:02:57 AM
Quote from: spetrillo on January 22, 2022, 01:17:22 AM
I created a VSS for each vlan
On a stand-alone ESXi host you generally create a virtual switch per physical switch. You define VLAN's across your entire site and trunk the VLAN's that connect to virtual machines to your ESXi.
To maximise aggregate throughput and availability, you can have more than one uplink between your vSwitch and your physical switches. A single link is fine for a home system where you'll likely have more single points of failure and won't generate enough traffic to saturate the link.
In your case, create one vSwitch and configure your external switch to tag the four VLAN's to the ESXi port (five if your WAN is a VLAN in the trunk). Configure port groups with the same VLAN on the vSwitch and test.
Bart...
Since I have 4 physical NICs for the LAN traffic, split by traffic type(Mgmt, WiFi, Streaming, Server) would I create one vswitch and add all the NICs to that vSwitch? Then I would tag all the vms as 0 and let the switch ports determine the vlan?
Quote from: spetrillo on January 22, 2022, 03:15:09 PM
Since I have 4 physical NICs for the LAN traffic, split by traffic type(Mgmt, WiFi, Streaming, Server) would I create one vswitch and add all the NICs to that vSwitch? Then I would tag all the vms as 0 and let the switch ports determine the vlan?
There are no VLAN tags on the VM's. Their VLAN membership is determined by the port group number of their vNIC. The VM OS has no configuration for VLAN at all. You can set one of its interfaces to another VLAN simply by picking another port group in the vNIC dropdown in VM settings.
The physical switch ports or the host physical NIC's do not determine the network. If you have VLAN's, the network is determined by the VLAN number. You can use multiple connections between the vSwitch and the physical switch like described here: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.networking.doc/GUID-D34B1ADD-B8A7-43CD-AA7E-2832A0F7EE76.html
Bart...
Quote from: bartjsmit on January 22, 2022, 07:59:01 PM
Quote from: spetrillo on January 22, 2022, 03:15:09 PM
Since I have 4 physical NICs for the LAN traffic, split by traffic type(Mgmt, WiFi, Streaming, Server) would I create one vswitch and add all the NICs to that vSwitch? Then I would tag all the vms as 0 and let the switch ports determine the vlan?
There are no VLAN tags on the VM's. Their VLAN membership is determined by the port group number of their vNIC. The VM OS has no configuration for VLAN at all. You can set one of its interfaces to another VLAN simply by picking another port group in the vNIC dropdown in VM settings.
The physical switch ports or the host physical NIC's do not determine the network. If you have VLAN's, the network is determined by the VLAN number. You can use multiple connections between the vSwitch and the physical switch like described here: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.networking.doc/GUID-D34B1ADD-B8A7-43CD-AA7E-2832A0F7EE76.html
Bart...
That document really talks to teaming of NICs, which is not what I am doing. As mentioned in my last post each vNIC/NIC is a separate vlan, which then equates to separate physical ports on my physical switch. I was creating a virtual switch to correspond to each vNIC/NIC/Switch port.
What is the make and model of your physical switch?
I have a Netgear 48 port GS728TS switch. I can configure it for vlans on the ports needed, whether tagged or untagged. What I have learned already is let the vlan on the virtual switch be 0 and do any vlan config on the physical switch.
There is nothing to stop you using untagged VLAN ports on the Netgear and splitting your networks by port/NIC/vSwitch the way you started to configure your infrastructure.
However, it is not very flexible. Let's say you upgrade your WiFi to AP's that support different SSID's linked to different VLAN's (e.g. Ubiqiti or TP-Link gear) that let you create extra wireless networks, let's say IoT and guest SSID's. Without trunking, you need to add physical network interfaces to your ESXi host and cable them to additional switch ports to let OPNsense manage these traffic flows.
I use Unifi and Netgear switches with Unifi AP's. Happy to share my config details by PM :)
Could I set the 4 ports on the physical switch as trunk ports, create one virtual switch that would include the 4 vnics/physical nics connected to the 4 switch ports, and then just create port groups for the needed vlans? If yes does ESXi support LACP or static LAGs?
Quote from: spetrillo on January 25, 2022, 12:58:18 AM
Could I set the 4 ports on the physical switch as trunk ports, create one virtual switch that would include the 4 vnics/physical nics connected to the 4 switch ports, and then just create port groups for the needed vlans? If yes does ESXi support LACP or static LAGs?
Yes, yes, yes, yes and yes :)
The LACP option requires a Distributed vSwitch which needs vSphere Enterprise+ licensing.
Bart...
Yes I saw the LACP option is only with vDS...so no worries there for now.