[Resolved] Another Question about No Internet On VLANs

[Reactor8393]

I'm trying to understand why my VLANs are not able to reach the internet.
All of the VLANs obtain the appropriate IPs from the Opnsense DHCP server.

LAN is the only network that can reach the internet. All other fail.
I'm also unable to see any traffic (Block/Pass) in the Firewall > Log Files > Live View from Home or Server Interfaces.

What am I missing?

OPNsense 24.7.7
Set DNS
System > General > DNS servers: DNS1 - 208.67.222.222, DNS2 - 208.67.220.220
Created VLANs
Interfaces > Other Types > VLAN igc1_vlan11 tag 11
Interfaces > Other Types > VLAN igc1_vlan12 tag 12
Assigned to Interfaces
Interfaces > Assignments: opt1 igc1_vlan11 parent igc1
Interfaces > Assignments: opt2 igc1_vlan12 parent igc1
Enable Interface and Set to Static IPv4
Interfaces > Home > Enable Interface
Interfaces > Home > IPv4 Configuration Type: Static IPv4
Interfaces > Home > IPv4 address: 10.5.11.1
Interfaces > Servers > Enable Interface
Interfaces > Servers > IPv4 Configuration Type: Static IPv4
Interfaces > Servers > IPv4 address: 10.5.12.1
Enable DHCP for Home and Servers
Services > ISC DHCPv4 > Home > Enable DHCP
Services > ISC DHCPv4 > Home > Set Range 10.5.11.50 - 10.5.11.250
Services > ISC DHCPv4 > Servers > Enable DHCP
Services > ISC DHCPv4 > Servers > Set Range 10.5.12.50 - 10.5.12.250
Enable Firewall Rules for Home and Servers
Firewall > Rules > Home: Pass, Interface: Home, Source: Home net, Any, Any
Firewall > Rules > Servers: Pass, Interface: Servers, Source: Servers net, Any, Any

RESOLVED!
Final edit:
I feel like I did everything the same expect I configured the device and made sure it was functional and added the VLANs afterwards. This is what I did and I hope it helps someone else.

In my Samba AD DC Bind configs I added the ACL in /etc/bind/named.conf.options for my VLANs:

Code: [Select]

acl goodclients {
10.5.10.0/24;
10.6.10.0/24;
172.16.55.0/24;
172.16.60.0/24;
172.16.61.0/24;
172.16.62.0/24;
};

options {
……
allow-query {goodclients;};
}

OPNsense 24.7.8-amd64
Interfaces > Other Types > VLAN
Device: vlan01
Parent: igc3
VLAN tag: 60
VLAN Priority: Best Effort
Description: Server VLAN60

Device: vlan02
Parent: igc3
VLAN tag: 61
VLAN Priority: Best Effort
Description: Server VLAN61

Device: vlan03
Parent: igc3
VLAN tag: 62
VLAN Priority: Best Effort
Description: Server VLAN62

Interfaces ServerVLAN60, ServerVLAN61, ServerVLAN62
Enable Interface
Set Static IPv4

Services > ISC DHCPv4
Enable ServerVLAN60, ServerVLAN61, ServerVLAN62
ServerVLAN60, ServerVLAN61, ServerVLAN62 – Set DHCP Range

Firewall > Rules
ServerVLAN60, ServerVLAN61, ServerVLAN62
Action: Pass
Quick: Enabled
Interface: ServerLAN60
Direction: in
TCP/IP Version: IPv4
Protocol: any
Source: ServerVLAN60 net
Destination: any

Sodola 4 Port 2.5G with 2 SPF
Configuration > VLAN > 802.1Q VLAN
VLAN 60
Port 1 – Tagged – Goes back to Opnsense
Port 2 – Untagged – Goes to PC
Port 3 – Tagged – Goes to other switch
Port 4  - Tagged - Goes to TP-Link TL-SG108PE switch
Port 5 – Not Member – No SPF
Port 6 – Not Member – No SPF

VLAN 61
Port 1 – Tagged – Goes back to Opnsense
Port 2 – Not Member
Port 3 – Tagged – Goes to other switch
Port 4 Tagged - Goes to TP-Link TL-SG108PE switch
Port 5 – Not Member – No SPF
Port 6 – Not Member – No SPF

VLAN 62
Port 1 – Tagged – Goes back to Opnsense
Port 2 – Not Member
Port 3 – Tagged – Goes to other switch
Port 4 Tagged - Goes to TP-Link TL-SG108PE switch
Port 5 – Not Member – No SPF
Port 6 – Not Member – No SPF

TP-Link TL-SG108PE 8 Port
VLAN > 802.1Q VLAN
VLAN ID 60
Port 1 - Tagged
Port 8 – Untagged
VLAN ID 61
Port 1 -Tagged
Port 7 – Untagged
VLAN ID 62
Port 1 – Tagged
Port 6 – Untagged

VLAN > 802.1Q PVID Settings
Port 6 – 62
Port 7 – 61
Port 8 – 60

[viragomann]

Are the devices able to resolve host names?

Is your outbound NAT in automatic or hybrid mode?
If so does it show automatically generated rule for the VLANs?

[Reactor8393]

Are the devices able to resolve host names?

No, I'm not able to resolve hostnames.

Is your outbound NAT in automatic or hybrid mode?

Outbound NAT is in automatic mode.

If so does it show automatically generated rule for the VLANs?

There are two automatic rules

Interface: WAN
Source Networks: Home networks, LAN networks, Loopback networks, Servers networks, 127.0.0.0/8
Source Port: *
Destination: *
Destination Port: 500
NAT Address: WAN
NAT Port: *
Static Port: YES
Description: Auto created rule for ISAKMP

Interface: WAN
Source Networks: Home networks, LAN networks, Loopback networks, Servers networks, 127.0.0.0/8
Source Port: *
Destination: *
Destination Port: *
NAT Address: WAN
NAT Port: *
Static Port: NO
Description: Auto created rule

[viragomann]

No, I'm not able to resolve hostnames.

So you have to resolve this issue.

What is the requested DNS server in the VLANs?

Normally the OPNsense DHCP server pushes the respective interface IP to the clients for DNS and Unbound DNS is enabled and listening on all interfaces.

Since you said, you have added rules to allow access to any, DNS requests should be permitted.

Run nslookup or dig on the devices to find out, which DNS server is requested. What do you get?

Ensure that the requests are not blocked by OPNsense firewall.

[Reactor8393]

Run nslookup or dig on the devices to find out, which DNS server is requested. What do you get?

Ensure that the requests are not blocked by OPNsense firewall.

The nslookup and dig returns the DNS server of the inferface IP 10.5.11.1 or 10.5.12.1.
I tried changing the DNS server to my local DNS server and times out.

When I view the logs in Live View, I'm not able to see any traffic from the Home or Server interface.
I also use the filter: interface, contains, Home - no success.

I also created a new interface with a different IP scheme, assigned it to a physical interface without VLANs, created the DHCP services, and created the Firewall rule of Any, Any - it works without issue.

Perhaps it's my equipment.

[viragomann]

The nslookup and dig returns the DNS server of the inferface IP 10.5.11.1 or 10.5.12.1.

And does it resolve successfully?

I tried changing the DNS server to my local DNS server and times out.

So you have a different local DNS server running?

If so you have to state its IP in the DHCP server settings.

> Perhaps it's my equipment.

Maybe it's not VLAN capable. For some network cards there are special drivers available.

[Reactor8393]

And does it resolve successfully?

It does not.

So you have a different local DNS server running?

If so you have to state its IP in the DHCP server settings.

I want to separate the DNS services.
I did set the IP in the DHCP settings in Services > ISC DHCPv4 > Home - no success.

I want to continue using Opnsense so I'm going to abandon the VLAN issue for another day or when I get better equipment.
For now, I'm going to separate my network based on physical ports.

Thanks for replying and trying to help me through the issue.

[EricPerl]

IMO, try to get the basic configuration (with unbound) running first, and build from there.

First off, IIRC, default rules don't generate logs by default, but you can change that in the firewall settings.
It's useful when you want to troubleshoot.

The TP-link easy managed switches are not particularly user friendly with regards to VLANs.
You need to use the 802.1Q mode.
IIRC (it's been a while), for a port you want to "assign" to VLANX, you need to set the PVID to VLANX, the port should be a member of only VLANX, untagged. Essentially VLANX stripped on egress, added on ingress.
The uplink port should have PVID 1 and be a member of all VLANs, tagged.

You might have gotten that right if DHCP worked though... but your diagram didn't match.

[Patrick M. Hausen]

The uplink port should have PVID 1 and be a member of all VLANs, tagged.

IMHO the uplink port should have a PVID that is not used anywhere, like e.g. 99, so you can run all VLANs including 1 tagged.

[EricPerl]

Hmm, isn't that only true if you want all traffic tagged (besides access ports)?
I gathered from a prior post that this is your preferred configuration.
Here, I've assumed an untagged native network (which also happens to be my configuration at home).

[Patrick M. Hausen]

An access port is an access port and a trunk port is a trunk port.

Only tagged frames on a trunk.

30+ years of experience managing data centres.

[EricPerl]

I respect the experience and appreciate the guidance.
I learned about this possibility last week (from you). I'm not there yet.

With my current gear (all TP-link Omada), I would expect this to be at best painful.
I already had planned to retire my router (next on my to-do, pending experiments while only a portion of my network is moved over) and it somehow will simplify the MGMT VLAN setup.
I'll have to refresh my memory about the known issues about future onboarding of new devices and if I see a path forward, I'll give it another go.
As of now, my management network is the native untagged physical network.

[Patrick M. Hausen]

We also run Unifi at the office. I simply dedicate a physical untagged interface on OPNsense to Unifi VLAN 1 and run only tagged VLANs on trunk ports.

In the data centre with Cisco gear and no auto provisioning VLAN 1 is not an issue.

[dstmnd89x]

Just wanted to know if this was ever solved...? I don't mean to thread jack, but I've joined up for this exact issue. Native LAN has no issues with net access but VLANs is another story. Why?

-DHCP and DNS working on all VLANS, clients getting IPs
-No rules blocking TCP/UDP on VLAN interfaces
-Unbound DNS enabled and listening on all interfaces
-Hybrid NAT mode w/ rules for each VLAN (checked auto rules)
-Restarted all services

I've been trying to figure it out on and off for weeks now. I'm no expert network builder, but I've worked with VLANs before and I can't seem to isolate the issue and fix it.

[EricPerl]

I think the OP gave up on VLANs for now. He had his own DNS servers and couldn't get past that hurdle.

Your setup seems simpler.
Personally, I've never messed with Outbound NAT rules and automatic rules have been sufficient so far.

On the FW: Only LAN gets an automatic in FW rule.
When I've brought up VLANs requiring Internet access, I've added:
1/ in, *, VLAN net, *, !RFC918 (an alias), * # at least as a start, you can refine further.
2/ in, TCP/UDP, VLAN net, *, This Firewall, 53(DNS)
Apply.

You can check the FW live view to see what's going on.