Run OPNsense virtualized and handle all traffic for the host and it's VMs?

[temporaryuser]

Hi everyone,

I want to install a virtualization OS (host) on a bare metal server, which is going to run a couple of virtual machines (VMs) which are going to function as server services (e.g. webservers, fileservers, etc.).

Now, since I would like to protect those VMs and be able to regulate the traffic from and to them, reach them via VPN, etc. I would like to have a firewall set between them and the internet, i.e. OPNsense.
But: Since I have only this one bare metal server at my disposal, I was thinking about installing OPNsense as a VM, too, instead of placing a second bare metal server with OPNsense between the host server and the internet.

Yes, I know, a virtualized firewall based on virtual NICs and VLANs is not as secure as a bare metal one, no doubt about that. But since I do not have the option for a bare metal server in this particular case, I am trying to at least improve security, instead of just having the host and it's VMs being totally exposed to the internet.

My questions: Is it possible and practically manageable to install OPNsense as a VM of the host and:
a) have the OPNsense handle/route all the traffic to and from the other VMs?
b) receive and manage all traffic from the internet coming to the host solely with the OPNsense VM, without touching the host first?

I would be happy about any feedback, thoughts and ideas about this! Has anybody done something similar?

Bye,
temporaryuser

[phoenix]

I don't have any problem running OPNsense in a VM on my ESXi server, why do you think a virtualized firewall is not secure?

[temporaryuser]

Hi Bill,

I don't have any problem running OPNsense in a VM on my ESXi server

Ok, and do you use this OPNsense VM to route/handle all traffic from/to the other VMs on that particular ESXi server? If yes: is that easy to set up and manage or what is your experience with such a setup?

why do you think a virtualized firewall is not secure?

I did not say that it is "not secure", but "not as secure" as a bare metal installation. The reason for this is IMHO that virtualization adds additional layers to the stack, i.e. the virtualization host, the virtualized NIC, the VLANs, etc. which add possibilities for additional vectors of attack (e.g. bugs/exploits in the VLAN stack) and / or configuration errors (e.g. getting overwhelmed with VLAN complexity), etc.

Cheers,
temporaryuser

[phoenix]

Yes, I use OPNsense VM for all the VMs on my host and all the other machines in my LAN.

IMO, there's no such thing as "not as secure as" - that means not secure to me. Security is a multi-layered approach and relying on a firewall or one single point of protection is self defeating - if the firewall is breached then you have problems. I do as much as I can on the firewall with IDS/IPS etc., etc. and add additional security measures on the machines in my LAN

[temporaryuser]

Yes, I use OPNsense VM for all the VMs on my host and all the other machines in my LAN.

Great! Maybe you can help me out with those questions:

• Is the logic of setting up all those virtual NICs on the host and then having the VM with the firewall handle all the traffic from and to the other virtual NICs/ the virtual bridges of the other VMs something difficult to set up or was it pretty straight forward for you?
• Are your other VMs totally isolated from the internet having the firewall VM handle all the routing?
• Is your host totally isolated from the internet having the firewall VM handle all the routing from and to it, or does your host continue to be reachable from the outside?

IMO, there's no such thing as "not as secure as" - that means not secure to me.

I disagree about that. Since there is nothing as 100,00% security, there must be gradients of security between 0% and 100%...

Security is a multi-layered approach and relying on a firewall or one single point of protection is self defeating - if the firewall is breached then you have problems. I do as much as I can on the firewall with IDS/IPS etc., etc. and add additional security measures on the machines in my LAN

I fully agree!

Cheers,
temporaryuser

[phoenix]

Assuming you don't have an Exterprise Plus licence then you're left with using the standard vSwitch. Create two of those and attach your NICs to them - if this is a home LAN then one NIC on each of the switches should do. Create the OPNsense VM with two NICs, connect one of those NICs to the 'WAN' vSwitch and the other to the 'LAN' vSwitch. Install OPNsense and configure it to your requirements, when it's up and running you should have a working firewall and LAN connection.

Nothing in my environment is directly connected to the internet (excpet the ESXi NICs) and everything (including the ESXi host) is routed throiugh OPNsense. Create any required VMs with single NICs (you don't really need more) and connect them to the LAN vSwitch and that should give you a quick and simple set-up, the beauty is you can rearrange things later should you desire to do so.

[temporaryuser]

Great, phoenix, thank you very much for your help! Sounds pretty much straight forward and a solid solution. I will do it as you told me. Thank you!

Cheers
temporaryuser

[phoenix]

You're welcome. :)

If you don't have much experience with ESXi then I'd suggest you be prepared to make mistakes, it is straight forward but it's still a pain when things don't work out as you expected. If you're using the free (or even a paid-for) version then you might find this little tool a boon for managing your ESXi host from a browser: https://labs.vmware.com/flings/esxi-embedded-host-client

What that fling does is install a web browser on the host and all the management functions are written in html and you don't even need a windows machine to run the VMware client.

[temporaryuser]

If you don't have much experience with ESXi then.. <snip>

Thank you very much for your additional help on ESXi, but we use a different, free (https://en.wikipedia.org/wiki/Free_software) virtualization operating system (which, by the way, comes with a web interface right out of the box).
The hints that you had given me before concerned ESXi, but this was not a problem for me since I understood the steps that you outlined and I know how to perform them in the virtualization environment that we use!

Please don't worry, I am sure, that your additional kind help for ESXi's web interface was not wasted since someone else using ESXi and reading this post will surely benefit from it!

Thank you very much for your help and time!
Cheers
temporaryuser

[Zeitkind]

If you are using proxmox, it has a small built in firewall which will be OK for most simple tasks.

[temporaryuser]

Hi Zeitkind,

If you are using proxmox, it has a small built in firewall which will be OK for most simple tasks.

Thank you very much for this information!

Cheers
temporaryuser

[cdburgess75]

OR why not have opnsense act as the hypervisor? :) 

[weust]

Because OPNsense is a firewall/router and not a hypervisor?

[cdburgess75]

Are you asking me or informing me of that :)

Imagine, maybe it can be a hypervisor with a extremely cool firewall.  The performance is definitely there for BSD's new virtulization software:

https://b3n.org/vmware-vs-bhyve-performance-comparison/


I get your point, truly I do.  Everyone will say, NO, you can't do that!  A firewall should never take these types of roles.  However, they are being sold as routers and switches with layer 1,2,3,4 and more of the OSI everyday being jammed into residential and commercial products everyday. So the tradition is fading a bit.  Haha,  lets do it man. 

[franco]

bhyve is in FreeBDS 10.x and available in OPNsense since its first version. If users want to use it it can make sense. The firewall runs the bare minimum config, namely the VMs with the services and the firewall rules between them plus connectivity to clients. In these setups the firewall doesn't run anything it doesn't strictly need so the VMs are safe to be extended in any possible way (risky services included). Good thing is the firewall itself isn't compromised by a bad exposed service this way.

I heard that this will also be part of pfSense 2.3 when it comes out. Can't have been wrong to ship it back in 2015 as there seems to be more demand for it than expected. :)