2 NICs - data in on one and out the other

[tdubbed]

The title of this posting confuses me, but this is the best I can do. I've tried searching this topic for days with varying degrees of success...

I have a Proxmox environment set up on bare metal on my server, and I have an OPNsense VM. Currently, OPNsense is configured correctly, and VIRTUAL hosts behind it can ping and have internet.

I have 2 physical NICs in my server. Since the first NIC accepts incoming traffic and forwards to VMs, I need to find a way to have traffic exit the second PHYSICAL NIC so I can attach my managed switch and give IP addresses to hosts attached to it.

I may have some more configuration to do in Proxmox, but assuming I don't, what factors do I have to consider when it comes to OPNSense? Firewall rules? Currently, plugging a computer into the second NIC gives an "Unidentified Network" message and an APIPA address.

I'm slowly crawling my way to solution after solution, and I'm learning along the way.

I appreciate anyone's assistance ahead of time.

Thank you!

[dev4openid]

Hi,

consider following this https://kb.protectli.com/kb/opnsense-on-proxmox-ve/

In addition, do NOT unless you are doing a lab, run this combo (Proxmox and OPNSense) in a production or home network. Place OPNSense on metal. There is strong advice to not do this.  Your call but understand the risks.

Cheers.

[tdubbed]

Oops!

[tdubbed]

Hi,

consider following this https://kb.protectli.com/kb/opnsense-on-proxmox-ve/

In addition, do NOT unless you are doing a lab, run this combo (Proxmox and OPNSense) in a production or home network. Place OPNSense on metal. There is strong advice to not do this.  Your call but understand the risks.

Cheers.

I appreciate the help! I have gone through this document, and everything is set up correctly as far as I can tell. This is NOT for production or home network. I will certainly use bare metal when the time comes.

As far as what I'm seeing, here is an image that explains what I'm seeing in detail. A Windows laptop is plugged into NIC #2, but I get APIPA address, "Unidentified Network". VM details for NICS are below that.

Got any ideas? Thank you!

[cookiemonster]

Do you want to use virtual nics on your OPN Virtual Machine or do you want to pass them through if you can?
Also you don't need to set Firewall=1 to those interfaces unless you plan on having two firewalls in place. Makes it more complicated. I suggest you don't when you start learning the concepts.
Additionally you seem to have a VLAN tag set on one of those interfaces. Perfectly doable but it requires that you manage your VLANs outside OPN, so it requires another device to route the traffic between a managed switch and the rest of your infra, including OPN.

[tdubbed]

Do you want to use virtual nics on your OPN Virtual Machine or do you want to pass them through if you can?

I have a Cisco managed switch that I would like to connect to the second physical NIC. I'm not sure if that answers your question or not. Apologies! I'm trying to wrap my head around all of this.. 

Also you don't need to set Firewall=1 to those interfaces unless you plan on having two firewalls in place. Makes it more complicated. I suggest you don't when you start learning the concepts.

Okay, great! I will remove that.

Additionally you seem to have a VLAN tag set on one of those interfaces. Perfectly doable but it requires that you manage your VLANs outside OPN, so it requires another device to route the traffic between a managed switch and the rest of your infra, including OPN.

Okay, this makes sense! I will play around with removing the VLAN in Proxmox and see what that does.

Thank you for giving me some food for thought! This REALLY helps!

[cookiemonster]

Do you want to use virtual nics on your OPN Virtual Machine or do you want to pass them through if you can?

I have a Cisco managed switch that I would like to connect to the second physical NIC. I'm not sure if that answers your question or not. Apologies! I'm trying to wrap my head around all of this.. 

That is the physical connection, that is fine. What I mean is that in Proxmox and by that linked tutorial, the VM is given a virtual interface that maps to the physical device. Another option is to "pass through" the device, which means there is no virtual nic created.
Depending on which way, the setup varies in that there is no linux bridge to create in Proxmox.

[tdubbed]

Do you want to use virtual nics on your OPN Virtual Machine or do you want to pass them through if you can?

I have a Cisco managed switch that I would like to connect to the second physical NIC. I'm not sure if that answers your question or not. Apologies! I'm trying to wrap my head around all of this.. 

That is the physical connection, that is fine. What I mean is that in Proxmox and by that linked tutorial, the VM is given a virtual interface that maps to the physical device. Another option is to "pass through" the device, which means there is no virtual nic created.
Depending on which way, the setup varies in that there is no linux bridge to create in Proxmox.

Okay, this makes sense a little! If I "pass through" the device, does that affect any other VM as far as networking goes? I would probably prefer to pass through. At least, it seems more straight-forward... Thanks!

[tdubbed]

Note to self: DO NOT PASS-THROUGH THE WRONG NIC.

LOL.

I immediately lost contact with my Proxmox server. I figured out how to rectify thanks to other people who suffered the same fate. Proxmox is back up, and I'll pass the CORRECT NIC now!

[tdubbed]

HOLY CRAP IT'S WORKING!!!!!! THANK YOU SO MUCH!!!!

My Windows laptop has an IP address from the subnet I require!

YEASSSSSSSSSSSSSSSSSSS!!!! You've saved the day, Mr. Monster. You deserve all the cookies you can handle. 5 days of poking at this!!

[tdubbed]

Okay, side note. My Ubuntu VM no longer gets an IP address from the virtual bridge. Is that part broken for good? It's fine if it is, but am I able to use the bridge as well as pass-through? Is that what a VLAN can accomplish?

Right now, the Ubuntu VM is attached to the vmbr0 obtaining DHCP from my home LAN network.

Thanks again!

[cookiemonster]

It will when it is set to get its ip from the OPN LAN instead.
Thanks for the cookies.