request for help with: single public IP, a bridge, two opensense-fw VM > VMs

[JL]

hey

thanks for taking a little bit of time to share your thoughts


I have this server at my disposal yet just one public IP

The server is a dual CPU 8c/16t with plenty of RAM and disk

the set-up i have in mind is    [ pubic IP] > [virbr0, virbr1, virbr2] > ( opensense-fw-1, opensense-fw-2) > virtual-LAN > VM1...N
on VM1..N there will be just a few VM running services

so, now  i have the public IP to which i configure DNS to resolve and i want to have this traffic arrive at both of VM1..N on different ports

to this end i expected to use the public-IP a a VIP-WAN but now i' m not certain if the ssh service running on the VM-host will still be reachable if i do so

or for that matter, if i could have the opnsense-ha-cluster correctly resolve the DNS and match with the hosts behind the NAT

[an_ipmc]

Hello.


From what i could understand from your question, with a single server and not manageable switch i would go this way:

1 - Public IP/ISP-Network Equip -> Dedicated NIC/Vswitch on your VMHost

2 - Opnsense/Firewall VMs
-> One virtual nic attached/connected to the dedicated NIC/Vswitch on your VMHost
( They would be your WAN interface on both VMhosts, for HA you can use RFC 1918 IPs and do a CARP with your public IP so you can get hardware high availability)
-> One or more virtual nic attached to your lan(s)/opt(s) that need internal routing/internet access

3 - For the DNS/PortForwardSsh/NAT/Whatever stuff with different ports/destination hosts, you gonna need to create alias and nat/firewall rules according to your requirements.
PF/Opnsense can do it all.
Take a look at: https://docs.opnsense.org/manual/nat.html
                      https://www.openbsd.org/faq/pf/nat.html

Make sure your firewall is the default gateway for the hosts/vms the nat rule is pointing traffic, or it will not work.

Since you`re exposing services/ports to the interwebz, some kind of ip banlist is recommended for some extra/added security:

https://docs.opnsense.org/manual/how-tos/edrop.html

[JL]

Thanks for sharing. That's roughly how i'm going about it.

By now i've found renting an extra public-IP is affordable and i've assigned this extra public IP to a bridge interface which is now exposed to the VM as a routed network interface (qemu/KVM)

The opnsense-VM appear to be running as expected in HA mode using carp. Now i want to add the IP assigned to the bridge interface as a HA IP to which i can bind various services.

so the set-up is now: [ public IP #1 ]-[ eth0 ] -> [bridge]-[public IP #2] -> [ opnsenseVM]

the PIP#2 is reachable from the internet but the traffic does not show in opnsense-VM

i understand this is becasue the PIP#2 responds to the external traffic arriving over PIP#1 but i do not understand in what set-up PIP#2 is 'owned' by the opnsense-VM cluster

[an_ipmc]

I don`t understand why you`re using a bridge at all. So this is just guesswork.

Maybe the problem is here: [ public IP #1 ]-[ eth0 ] -> [bridge]-[public IP #2] -> [ opnsenseVM]

Your Eth0 is the physical interface attached to opnsense VMS.
There is no need to use any routed interface on your VM host since your opnsense firewall/VM is the network default gateway.

Maybe you`re using the bridge interface  for VM host management... but:

It would be best if you were controlling/filtering all of your traffic by using your Firewall -> This is a security best practice and also a global collaboration for the interwebz hygiene:)
Configure your firewall to filter it all, and use some VPN (Wireguard is your friend) for more secure/controlled management access.

In guesswork mode, I would exec the configuration this way:

1 - Configure dedicated VM host interwebz interface as WAN on opnsense VMs with RFC 1918 IPs ( 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 ) - Don't forget to go to Opnsense GUI and deselect the box that blocks private networks cause you`re using them for your setup with carp to work

2 - Go to Opnsense GUI -> Interfaces -> Settings -> Virtual IPs and Configure a CARP Interface with your first Public IP.

3 - Go to Opnsense GUI -> Interfaces -> Settings -> Virtual IPs and Configure your second Public IP as an IP Alias of your WAN Interface and select your WAN interface CARP VHID

4 - Configure your LAN/Opt interfaces

5 - Run a tcpdump on your WAN/Lan interfaces to confirm traffic is flowing

6 - Setup a Wireguard VPN for VMHost Management
 
7 - Create your Aliases/PortForward/NAT/Firewall Rules to redirect services to the correct LAN Hosts/Targets