Looking for a practical worked example for starting with VLANS

[sparticle]

Best make that into 2 ports.

Make 2 LAN ports in OPNSense?

Or make 2 physical connections to the main network switch?

Cheers

[bimbar]

Both, one interface for vlan 1 untagged, and a second one for the vlan trunk (with dedicated cables to the main switch).

[sparticle]

I did think I was getting somewhere. But alas not it seems.

I have setup a laptop on a different empty switch Netgear GS724Tv4. This has a fibre uplink to the main HP switch.

I setup a new interface in OPNSense and created a VLAN 50 with DHCP service enabled following this tutorial.  https://www.zenarmor.com/docs/network-security-tutorials/how-to-configure-vlan-on-opnsense

All seems well at the OPNSense end and I can ping the OPNSense server on the new subnet 10.0.50.254 from the native LAN! Confused by this behaviour also.

At this point there is nothing configed on any switch they are running as a single LAN.

I tried setting Port 2 on the Netgear to PVID 50 VLAN 50, removing the untagged VLAN 1 from port 2 and adding VLAN 50 to the uplink port link to the HP. Then also adding VLAN 50 to the HP TRUNK uplink port and the OPNSense trunk port. So in theory this is what I thought would happen.

Laptop connected to Netgear port 2 all packets are Tagged with VLAN 50 by the switch. They are passed as Tagged to the HPon its TRUNK uplink port which also is in VLAN 50. Then it can get to the OPNsense server via the TRUNK connection to the ESXI which is set to 4095 at the ESXI end so will pass all VLAN traffic. Then the 10.0.50.0/24 subnet and interface will get the traffic and respond with a DHCP lease. The laptop gets an IP and all is well.

That fiction didn't happen. I cannot ping the 10.0.50.0/24 network from the laptop. There are firewall rules on the VLAN50 interface copied from the LAN interface to allow DHCP etc. So it should work exactly like the LAN interface which is the VLAN parent interface. Even if I config a static IP in the VLAN 50 subnet on the laptop I cannot get to the OPNSense server on it's IP 10.0.50.254.

I am still confused on the tagged and untagged configuration. I am obviously missing something fundamental. I have done so much reading on this I probably have some analysis paralysis!

As I understood it. If I want dumb equipment like a laptop to connect via a specific VLAN I need to set the port it is connected to as an access port. If it was a WAP that is VLAN aware it may need to be a TRUNK port with the VLAN tags set for the various wifi networks. But stick to a single laptop trying to connect to OPNsense VLAN 50 and getting network services like DHCP and DNS. Do I set the netgear port as tagged 50? What do I set on the uplink ports between the switches? There is nowhere to set the port as a TRUNK port in the netgear and a ton of less than clear conflicting info on the web. I can either TAG the uplink port into a VLAN or have the port included in the VLAN config as untagged. On the HP end I can set port 1 (OPNSense connection) and port 25 (uplink port) as TRUNK ports and I can again set the TAG for VLAN 1 and 50 or include these ports in both VLAN 1 and 50 untagged.

If I can get just one VLAN working properly I am sure I can use that to learn about intervlan traffic etc. which is something I will need to properly migrate from a single subnet to a fully segregated VLAN based network.

The plan is to have the following.

MANAGEMENT VLAN able to access anything across all VLANS
NETWORK SERVICES VLAN to provide email, NAS, Streaming services etc.
HOUSE AP VLAN(s) for the 6 AP's in the main house with multiple networks for IOT Devices, Adults, Kids, Guests.
OFFICE VLAN(s) 2 AP's with multiple networks for IOT Devices, Employees, Visitors.
OUTSIDE VLAN(s) 5 AP's with multiple networks for IOT Devices, Us, Visitors and Guests.

Multiple of the above would need to be able to communicate with each other for provision of services. e.g. The NETWORK SERVICES and IOT Devices would need to be able to talk to each other to stream content to phones/tv's etc.

But right now this is a dream as I can't even get a single simple VLAN to work. It feels like I am missing a critical piece somewhere that has just not clicked into my consciousness.

Advice, encouragement, and guidance appreciated.

Cheers

[sparticle]

I really need some help with this.

I have now a very simple setup.

The main OPNSense config is as it was with the addition of a single VLAN config. I restored the config from a previous point before I started messing with VLANS to ensure I was back at my base config for the network. I followed this guide https://www.zenarmor.com/docs/network-security-tutorials/how-to-configure-vlan-on-opnsense and setup the VLAN exactly as the LAN is configured but with a new subnet with the LAN interface as the parent. The new VLAN 50 interface OFFICE has DHCP services configured exactly the same as the LAN interface in the new subnet. e.g. 10.0.50.0/24 with an interface address of 10.0.50.254. I have cloned the firewall any rule from the LAN to the OFFICE net. Everything appears to be setup correctly. As I have an any rule on the LAN I can ping the OFFICE interface from outside the OPNSense server from my PC on the main switch.

On the HP Switch that OPNSense is connected to I have configured VLAN50 and the ACCESS and TRUNK ports to connect to OPNSense and the other switches. See attached image of the setup. This is a very simple setup to get one VLAN working. It doesn't work and I cannot get DHCP from OPNSense or even if I config a static IP in the OFFICE subnet I cannot ping the OPNSense OFFICE interface.

I am completely at a loss as to why this is not working. The VLAN config on the switch looks right. The OPNSense VLAN config looks right I have FW rules and DHCP and DNS services on the OFFICE VLAN.

In words the switch is configured as follows. See image for detail.

Port 1 TRUNK Untagged 1 Tagged 50 PVID 1 (LINK TO OPNSENSE)
Ports 11 and 12 ACCESS Untagged 50 PVID 50 (LAPTOP TEST PORTS)
Port 17 TRUNK Untagged 1 Tagged 50 PVID 1 (WAP with 2 wif networks 1 on the default VLAN and 1 on VLAN 50)
Port 25 TRUNK Untagged 1 Tagged 50 PVID 1 (LINK TO REST OF NETWORK)

This should work but it doesn't, OPNSense shows no packets on the OFFICE interface.

Can anyone please put me out of my misery and help me to get VLANS working.

Just to add I know the switch is working as I can config an admin address in the switch on the VLAN 50 subnet and I can ping it from the Laptop on the VLAN 50 network. So I know the switch ports as working as expected within the switch. I also know the switch VLAN config is working between switches. I can ping the HP on its VLAN50 address from the Netgear connected via a TRUNK to TRUNK connection to the HP oort 25 using the laptop manually configured with a VLAN50 ip.

BUT, I get the destination host unreachable and no route to host if I try to ping the OPNSense VLAN50 interface on 10.0.50.254. No packets are received on the OPNSense OFFICE (VLAN50) interface. Also the WAP on the HP TRUNK port 17 gets no DHCP service either. I can configure a static IP on the wifi connection and connect to the VLAN 50 wifi network but can't get anywhere.

It is like any VLAN subnet on the LAN interface is blocked and I suspect that pinging the VLAN 50 address from the default network is simply getting a response from the parent interface as stats show no packets on the VLAN 50 interface.

What is going on here?

Cheers

[bimbar]

Seems fine to me. Probably something simple and stupid, but those are the ones that are hardest to find.

[sparticle]

Seems fine to me. Probably something simple and stupid, but those are the ones that are hardest to find.

Thank you for replying but like what? What simple checkbox or config item can stop all VLANS from working on a parent interface? This is a simple single VLAN config attached to the parent LAN interface. It is like OPNSense is ignoring any VLAN tagging coming into the parent.

[bartjsmit]

Make sure the tag is coming into the parent. Do packet traces on the switch side (mirror port?) and the firewall side; Interfaces: Diagnostics: Packet Capture. Wireshark is your friend.

Try with a different switch. Five port managed switches likely cost less than the time you've spent on this already if you paid yourself minimum wage 

[sparticle]

I already have two perfectly good managed switches I am working with. As my OPNsense is virtualised in an ESXI VM there seems to be a lot of stuff out there about VLANS not working correctly with 6.7 vswitches and port groups with the VMXNET3 adaptor.

Timed out on this for a while whilst I make some more of that minimum wage

[Patrick M. Hausen]

if you are running virtualised you should create all VLANs in ESXi/vSphere and map each to an individual interface in OPNsense. Also E1000 is preferred over VMXNET3.

[sparticle]

So you seem to be saying I need to create a PG per VLAN attached to the vswitch. Then in order to use that create a new vnic in the OPNSense VM for each VLAN and attach it to the VLAN PG.

Then in OPNSense create the VLAN and assign it to the new VM vnic?

[Patrick M. Hausen]

So you seem to be saying I need to create a PG per VLAN attached to the vswitch. Then in order to use that create a new vnic in the OPNSense VM for each VLAN and attach it to the VLAN PG.

That is the recommended way, yes.

Then in OPNSense create the VLAN and assign it to the new VM vnic?

Not a VLAN - from the guest OS' point of view that is just a regular untagged interface. So you assign an interface and the create rules, DHCP, etc. as you would with VLANs. But all the switching fabric things happen in the vSwitch.

[sparticle]

Not a VLAN - from the guest OS' point of view that is just a regular untagged interface. So you assign an interface and the create rules, DHCP, etc. as you would with VLANs. But all the switching fabric things happen in the vSwitch.

AH OK I think I am starting to get it. You are saying that the new VM provided vnic is just another NIC in OPNsense that I can assign to a network I create called Guest for instance with an address and subnet configured for the VLAN like 10.0.50.254/24 then add DHCP DNS etc. to that interface and rules as before. At this point it is just a subnet. I would have connected that vnic to the PG for VLAN 50 in the VM settings in ESXI. All traffic in/out of the Guest NIC in OPNSense would be untagged until it gets to the PG which would tag it as VLAN 50(?),  is this correct ? I thought that PG's only allowed tagged VLAN taffic matching the VLAN ID set on the PG config. 0 default or VLANID or 4095 for all?

Currently, the one working VLAN50 is configured in OPNsense as per the guide and assigned to the LAN parent interface as VLAN50. The PG that the OPNSense talks to is set as 4095 and it works. Although I am now questioning that. As I believe setting 4095 on the PG means matches all VLANS.

I need to completely rethink this if your guidance is the right way to do this. As this is the start of a journey and I am keen to get to the destination the right way.

[Patrick M. Hausen]

You correct about the virtual NIC.

One might argue that the "best" way to configure all of this would be PCIe passthrough of an entire interface to OPNsense, then configure VLANs there.

You get better performance and can add/remove VLANs without shutting down the VM. Also adding of interfaces might reorder the existing ones, I read on this forum.

All traffic will move through your switch, though.

[sparticle]

You correct about the virtual NIC.

As I wrote multiple thoughts on the vnic which bit of what I wrote is correct?

AH OK I think I am starting to get it. You are saying that the new VM provided vnic is just another NIC in OPNsense that I can assign to a network I create called Guest for instance with an address and subnet configured for the VLAN like 10.0.50.254/24 then add DHCP DNS etc. to that interface and rules as before. At this point it is just a subnet. I would have connected that vnic to the PG for VLAN 50 in the VM settings in ESXI. All traffic in/out of the Guest NIC in OPNSense would be untagged until it gets to the PG which would tag it as VLAN 50(?),  is this correct ?

This?

[Patrick M. Hausen]

Yes, that part. OPNsense will not know about VLANs, only logical interfaces, networks, addresses ... the tagging and switching is left to the system that is far better at layer 2.

But with virtual NICs you will always have performance issues *if* you need to reach wire speed, i.e. at least a Gbit/s of throughput. In that case a passthrough NIC is best, IMHO. If your hypervisor host has got an unused NIC or two ...