VLAN Problems

[Gareth_H]

Hello

I have been messing around with OPNsense to build a homelab and sure I must be doing something wrong.

I have a proxmox server and an ubuntu desktop both in VLAN10, after not getting IP's from DHCP for ages, they have magically got them now.

However, none of the devices can access the internet or ping each other, or even the default gateway.
1. Switches are configured
2. VLAN created
3. Default LAN rule copied and assigned to VLAN10

I really am pulling my hair out, can anyone help, please?

Cheers

Gareth

[meyergru]

If both devices are on the same VLAN and in the same subnet, OpnSense is not even involved in the traffic between them.

Show your network topology and involved VLANs and subnets/gateways.

[Gareth_H]

Thanks

Please see here, hope that helps - https://photos.app.goo.gl/T48huMWPMr9pTkdU7

[EricPerl]

Please attach screenshots to your post. Some balk at external links...

[meyergru]

For one, I do not see where VLAN 20 is going through your switches. They only have VLAN 1 (untagged) and VLAN 10 configured.

And you did not post the actual IP settings of your Ubuntu Desktop and your Proxmox host.

Also, if your Ubuntu Desktop is configured as usual, it will be untagged, so your port 8 of your TL-SG608E should be untagged for VLAN 10, not tagged. You have not shown your Proxmox config, so probably the management interface expects VLAN 10 untagged as well.

Review your VLAN tags and your switch configurations. 

[Gareth_H]

Please attach screenshots to your post. Some balk at external links...

I posted the link because no matter what I do, I can't get the image to show up in the post :-(

[Gareth_H]

For one, I do not see where VLAN 20 is going through your switches. They only have VLAN 1 (untagged) and VLAN 10 configured.

And you did not post the actual IP settings of your Ubuntu Desktop and your Proxmox host.

Also, if your Ubuntu Desktop is configured as usual, it will be untagged, so your port 8 of your TL-SG608E should be untagged for VLAN 10, not tagged. You have not shown your Proxmox config, so probably the management interface expects VLAN 10 untagged as well.

Review your VLAN tags and your switch configurations. 

Hi

The link I shared had all of that in, but the image won't embed from Google Drive, Google Photos or even Imgur. So let me try and explain.
- I havent created VLAN20 yet as just trying to get the Proxmox host and Ubuntu Desktop working in VLAN10
- They are in ports 7 + 8 on a TP-Link TL-SG608E switch.
- Other devices on that switch are untagged (another PC, another server and an TP-Link Deco AP)
- Port 1 is the uplink port to TP-Link TL-SG108E switch
- VLAN10 has an IP of 10.100.74.1/24 - Ubuntu has got 10.100.74.50 from DHCP and Proxmox 10.100.74.51
- My configs are
 VLAN: 1, VLAN Name: Default, Member Ports: 1-6, Untagged Ports: 1-6
 VLAN: 10, VLAN Name: VLAN10, Member Ports: 1, 7-8, Tagged Ports: 1, 7-8

Upstream - TL-SG108E switch
- Port 1 comes from the other switch above
- Port 8 goes to OPNsense Box
- Other ports: 1 goes to another AP and 1 goes to a TP-Link TL-SG605E (all untagged) with a PC and Xbox and another AP.
- My configs are
 VLAN: 1, VLAN Name: Default, Member Ports: 1-8, Untagged Ports: 1-8
 VLAN: 10, VLAN Name: VLAN10, Member Ports: 1, 8, Tagged Ports: 1, 8

Other points
- LAN DHCP is 10.28.74.1/22 and everything works fine
- If I put ports 7 + 8 into the default VLAN, they get an IP from LAN DHCP and work without issus

Hope that helps.

[EricPerl]

Reply (not quick reply) or preview. Drag image below the edit box. That attaches the image to the post.
You cannot view this attachment.
This said, the insert dialog is messed up...

[EricPerl]

My comments are going to echo meyergru's

Unless you have configured proxmox and Ubuntu to expect tagged traffic, the switch ports should be access ports (PVID = 10, 10 Untagged, 1 not member).
That would be normal for the Ubuntu desktop (traffic is tagged as soon as it enters the switch).

Arguably, if you intend to use other VLANs on Proxmox, you may want to add a VLAN and move the management IP to vmbrX.N.
In that case, port 7 will have to become another trunk (all member VLANs tagged).

[Gareth_H]

Reply (not quick reply) or preview. Drag image below the edit box. That attaches the image to the post.
You cannot view this attachment.
This said, the insert dialog is messed up...

Thanks, Doh!!! :-)

[Gareth_H]

Thanks for all of the help everyone - It looks like I just had everything badly configured.

VLAN 10 all correct now - I think. Devices gets an IP and can get to the internet.

Although, I did have my firewall configured incorretly too :-)

Although I still dont get why the ports with VLAN10 are meant to be untagged.

[meyergru]

An untagged port on a switch for a certain VLAN means that the VLAN is stripped on egress and added on ingress. The switch itself always carries all VLAN tags on its internal backbone. You merely decide what is actually presented on a port. Most devices cannot handle VLANs and expect everything untagged, so you determine which VLAN there are on.

Normally, you even do not want that the end device tags packets by itself, namely for protection. If that were not so, any device could connect to any VLAN, thus reducing security. By having the port untagged, such devices cannot connect to a VLAN even if they tried.

The exceptions are devices that can or must actually handle VLANs themselves, like APs, switches and virtualisation hosts, where VMs with different VLANs can be present. Each of those VMs gets untagged traffic as well (unless it is an OpnSense VM).

[Gareth_H]

Thank You Anyway, all working now :-)

[EricPerl]

When VLANs are used for security, you kinda have to decide where tagging takes place, and which devices can be trusted.

You have to trust your network infrastructure (router, switches, APs if they are VLAN aware).
It's safer to handle VLAN management on these devices.
On switches, there ends up being 2 types of ports:
* Access ports connected to end devices that you don't necessarily have to trust. Tagging (traffic coming in the switch, with PVID) and Untagging (of Untagged VLAN for traffic coming out of the switch) takes place at the port.
* Trunk port connected to other network equipment. Most traffic should be tagged. You decide which VLANs can go through.

The recommendation is that all trunk traffic be tagged (eliminating untagged traffic within the network infrastructure) but you can't do this with your easy-smart switches because I don't believe they support a management VLAN (for their own IP you use to configure the ports).

This extends to virtualization hosts, which in facts extends the network (virtual network and machines).
For example, on Proxmox, using a VLAN aware bridge and assigning a VM to VLAN is essentially creating an access port for that VM.
The switch port connected to the corresponding physical interface should be configured as a trunk.

[Vilhonator]

First If you have some custom built PC running Opnsense, make sure the network interface supports IEEE 802.1Q (yes, for vlan to work, network interface your opnsense has must support it. Most mainstream consumer market interfaces like cheap realtek NICs don't have that, it is one of few things you have to pay bit more extra. Intel l350 series 1 gigabit NICs aren't that expensive and for 10Gb ethernet, x550 series is solid choice)


Secondly reboot your opnsense if you havent done so, after you finished setting things up and tried if it works. Rebooting is sometimes required for changes to take affect (this should be first thing you should try. I have had my share of similar vlans not working situations, and rebooting fixed the issue on vast minority of cases, but still something worth trying)

If that doesn't work, next make sure the switch port which is connected to opnsense is set to tagged mode for VLAN 10, secondly assign the port your computer is connected to as member or ACCESS mode to vlan 10 (opnsense uses 802.1Q tagged vlan, you have to set port(s) you connect to opnsense and other possible switches to trunk or tagged mode for VLAN 10 and ports you plan to connect your client devices to access mode or member mode for vlan 10 (I am familiar with Cisco terminology, tagged and untagged is something they don't use on hardware I have).

Your LAN belongs to different IP range, which is why you can't ping from LAN to VLAN10 and it is possible, it's a routing issue. to test this, run a traceroute on machine connected to vlan10, on linux system open terminal, and type "traceroute 10.28.74.1" and "traceroute google.com". If both fail, it's obvious routing issue,  easiest way to fix it, is to set LAN and VLAN10 to same private range, in your case, 10.28.74.1/22 for LAN and 10.28.75.1/22 for VLAN10

If that's the case, placing both networks to same private range should fix the issue (since opnsense creates route for LAN automatically by default).

Your switch supports IEEE 802.1Q so there shouldn't be issues there.

To summirize.

Make sure your opnsenses NIC supports IEE 802.1Q (802.1Q for short)

Make sure you have rebooted the system.

Make sure your switch have correct port(s) set as Tagged for VLAN 10 and correct port(s) assigned as member ports of VLAN 10 (I would assume untagged, but might be wrong).

Make sure routing is correct (you are able to traceroute to your VLAN gateway)