VLAN can't reach internet nor DNS

[SiliconOxide]

I'm trying to setup a guest vlan names GVLAN40, Vlan tag of 40. Client can get a DHCP address, reach the management interface, ping the gateway and DNS server. However, I can not get to the internet and can not resolve domain names. You will have to excuse me as I am new to vlans, so I anot 100% sure if I am setting it up correctly. The client received the correct DNS and Gateway. It looks like the traffic is being blocked by an automatic rule. I setup the firewall rules as I think they need to be done. I think I have attached most of the relevant settings in the images. I ope someone can offer some insight into my stumbling around.

Thanks,
Chris

[patient0]

The VLAN interface assignment itself looks ok, is 192.168.40.0/24 your VLAN subnet?

The firewall rules are a bit of a mess. The rules in general are "first match" (the yellow bolt sign), the first that matches will be applied and no further evaluation of rules is done. Plus they apply in direction 'in' (arrow to the right), traffic coming from the interface network to the router.

• 127.0.0.1 is the local address of a computer, that traffic will never leave a computer and therefore will never reach a firewall
• Then next two rules are 'Default allow all' for IPv4 and IPv6, nothing after these two rules is ever evaluated. As the name implies it allow access to everything

The firewall log entry shows that on the _LAN_ interface a client with the IP 192.168.40.21 wants to ping 9.9.9.9. That would indicate that the client is not correctly set to VLAN40.

How to do you connect your client to VLAN40? Do you have a switch where the client is connected to a VLAN40 access port? Or do you configure VLAN40 on the client itself?

[SiliconOxide]

Yes, 192.168.40.0/24  is the vlan subnet.

The 127.0.0.1 rules were taken from: https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/ to force the use of unbound and the ntp server on opnsense. I'll disable those for now for trouble shooting, but I do have the same rules on the LAN and they appear to work.

I'll got through and rearrange the rules, putting the two allow all rules at the bottom.

As for connecting to VLAN 40, I have a Mikrotik switch with ports tagged for Vlan 40 (opnsense port and wap port), connected to a Ruckus WAP with a wireless network assigned to VLAN 40. How is the client correctly able to obtain an IP using the vlan DHCP server, but not send traffic over the vlan? I did see the LAN interface noted in the auto block rule, but I assumed that that was correct because the vlan's parent is the LAN

[cookiemonster]

Don't be too surprised if there are strange behaviours. You don't seem to have a trunk from the mikrotik to OPN, instead you seem to have mixed tagged and untagged traffic arriving via mlxen0.- That is tagged 40 from the GVLAN40 and the untagged from VLAN.
You might have NO problems though until very much in the future when you forgot this and think "it's been working forever, OPN upgrade X broke things".

[patient0]

The 127.0.0.1 rules were taken from: https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/ to force the use of unbound and the ntp server on opnsense. I'll disable those for now for trouble shooting, but I do have the same rules on the LAN and they appear to work.

That does work for a port forward/redirect, yes. Because you replace the real destination IP with 127.0.0.1 which then would point to the router where the rule is evaluated.

But for a normal firewall rule that does not work. These rules match whatever traffic goes into an interface. And if you send a package from your client with destination 127.0.0.1, it will never even leave your client.

As for connecting to VLAN 40, I have a Mikrotik switch with ports tagged for Vlan 40 (opnsense port and wap port), connected to a Ruckus WAP with a wireless network assigned to VLAN 40

I'm rusty on the Mikrotik VLAN config and have no knowledge about Ruckus.

On the MikroTik you have the ports to OPNsense and WAP configured as trunks, carrying untagged traffic (with what PVID?) und tagged VLAN40?
Due to my missing Ruckus knowledge I would configure an additional port as an VLAN 40 access port (PVID to 40) and connect a client by cable directly. If that works move on to Ruckus, although if someone else would chime it would probably be faster.

@cockiemonster can help pinpoint the issue?

[cookiemonster]

sure we can help to check the trunk setup (might or not be "the issue").
From the mikrotik you need to set the "wire" going from it to OPN as a trunk. That means ALL traffic is tagged.
Then on the OPN side of it, you need to have all VLANs as tagged devices. The "parent" device does not need to be assigned.
See mine. igc1 is the "parent" and not assigned. The two VLANs hanging from it are. igc1 is the "wire" from the mikrotik switch as trunk.
You cannot view this attachment.

>I'm trying to setup a guest vlan names GVLAN40, Vlan tag of 40. Client can get a DHCP address, reach the management interface, ping the gateway and DNS server. However, I can not get to the internet and can not resolve domain names.

How do you want to setup your VLAN DNS? Normally you run on OPN either dnsmasq or Unbound. In Unbound you set it to listen to all interfaces, which means will start listening on your VLAN IP. Then you simply copy the DNS allow rule from your LAN.
DHCP obviously needs also setting up on the VLAN.

[SiliconOxide]

From the mikrotik you need to set the "wire" going from it to OPN as a trunk. That means ALL traffic is tagged.
Then on the OPN side of it, you need to have all VLANs as tagged devices. The "parent" device does not need to be assigned.
See mine. igc1 is the "parent" and not assigned. The two VLANs hanging from it are. igc1 is the "wire" from the mikrotik switch as trunk.

I was in the middle of replying asking if I should convert the untagged traffic on opnsense to tagged traffic when you posted that.

How do you want to setup your VLAN DNS? Normally you run on OPN either dnsmasq or Unbound. In Unbound you set it to listen to all interfaces, which means will start listening on your VLAN IP. Then you simply copy the DNS allow rule from your LAN.
DHCP obviously needs also setting up on the VLAN.

I am using unbound which is set to all interfaces already and have the allow DNS rule on the VLAN 40 FW "    IPv4+6 TCP/UDP    GVLAN40 net    *    GVLAN40 address    53 (DNS)    *    *       Allow access to LAN DNS server ". DHCP is setup on vlan40 as well and the client is grabbing the correct servers.

I'll convert the LAN to a vlan, setup the mikrotik port to opnsense as an only tagged trunk and go from there.

And just to clear up, on the mikrotik:  sfp port to open sense: tagged only, vlan 10(converted from LAN) and vlan 40. Port ether4 w/ruckus, access port vlan 10, tagged vlan 40. All other ports, access port vlan 10.

[EricPerl]

The 127.0.0.1 rules were taken from: https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/ to force the use of unbound and the ntp server on opnsense. I'll disable those for now for trouble shooting, but I do have the same rules on the LAN and they appear to work.

That does work for a port forward/redirect, yes. Because you replace the real destination IP with 127.0.0.1 which then would point to the router where the rule is evaluated.

But for a normal firewall rule that does not work. These rules match whatever traffic goes into an interface. And if you send a package from your client with destination 127.0.0.1, it will never even leave your client.

That was probably fine. These rules appear to be associated FW rules from PF rules.
In such cases, the destination is indeed the "destination IP" of the PF rule because the FW rule is evaluated after the PF rewrite of the destination.

Edit: the giveaway that they are associated rules from PF rules is the fact that the edit and clone buttons are missing.

[cookiemonster]

And just to clear up, on the mikrotik:  sfp port to open sense: tagged only, vlan 10(converted from LAN) and vlan 40. Port ether4 w/ruckus, access port vlan 10, tagged vlan 40. All other ports, access port vlan 10.

Yes managed switch to OPN is tagged only.
Ruckus, not much of an idea, I don't have experience with them. If it is a wireless access point, if it connects to an access port on the managed switch, then is to be as you configure the port on the switch, either with a tag if is the AP is VLAN-aware and tags traffic as it delivers it to the switch, and then the switch port needs configuring for that.

[cookiemonster]

p.s. you should also attach the pic of your NAT rules. We're missing that bit.

[SiliconOxide]

I setup vlan10 for the main lan and vlan40 for the guest lan. SFP+ 1 (to opnsense) set to trunk vlan10 and vlan40 tagged only, ether4 (Ruckus WAP) to vlan10 access/PVID10 and vlan40  tagged, the remaining ports to access 10, PVID 10. I then set OPNsense to vlan10 for the lan. Turned Vlan filter on on the bridge and rebooted the switch. the guest network now has internet and dns. OPNsense looks to be running 100%. I haven't noticed and issues so the network is working 100% for now.

However, not all sunshine and rainbows. I locked my self out of the switch. I thought I could access it by MAC address, but it looks like that isn't an option. I have a back up before I applied the vlan filter on the bridge. The vlans did have IP addresses assigned to them. So, before I reset the switch and apply the backup, how do I make the management interface accessible from vlan 10. I believe I may have made an error in the attached images, highlighted areas.

[cookiemonster]

the guest network now has internet and dns. OPNsense looks to be running 100%. I haven't noticed and issues so the network is working 100% for now.

Glad to hear.
As to your current questions, 'mafraid I can't tell.  No idea what those screenshots are even. Or what bridge that is referring to.

[EricPerl]

I suspect this is configuration of the Mikrotik switches. I don't own one so no clue either.
It looks complicated...

It's pretty typical for managed switches to be able to:
* Define a default network with a VLAN ID, or just a default VLAN ID corresponding to untagged traffic, instead of assuming it's 1.
When you have configured the router/switches/APs to only handle tagged traffic, it's not a bad idea to give it a value not used otherwise.
* Define a "management VLAN" which is the VLAN used by the switch for its own IP (static or DHCP), including getting config when it's under the umbrella of a controller, or allowing access to its GUI...
If you use a static IP, it'd better be in the range of the corresponding VLAN.
If you have not defined a management VLAN, management is done untagged and hopefully you have a port allowing untagged access.
If you have defined one, you need to use an access port for that VLAN or use FW rules on the router to allow inter-VLAN access.

[SiliconOxide]

After I recovered access to the switch the guest vlan(40) still wasn't working 100%. I thought what I did had fixed it, but I realized that using android and zorin linux it worked fine. But when I tried my windows 10 notebook again it did not work. No dns, no internet access. I had created another vlan for IOT devices. I hadn't used that vlan yet so I didn't configure much except the DHCP server and the allow all firewall rules. After fighting with the vlan40 guest I switched the vlan on the WAP for the guest network to the unused IOT vlan. The internet finally worked on the windows 10 notebook. After removing config items from the guest vlan to get it to match and trying after each change I narrowed the error down to the guest vlan having an IP on Mikrotik switch. Once I disabled that the guest vlan was working properly. See attached image.

The only thing I don't understand is why linux and android worked fine but win10 did not.

Thank you for your help.

[EricPerl]

I'm not familiar with the Mikrotik GUI but if these are IPs for the switch itself, that seems like a lot. Also 3 of them are in the same network.

For comparison, my switches are configured with:
* a "Default" interface with a subnet of my choosing but that's for untagged traffic and there's none. An unused VLAN ID is associated.
* a "management" VLAN, and for DHCP in that network (IP, subnet, GW obtained dynamically). In practice, there are DHCP reservations for these.
* a list of VLAN IDs
* per port settings with PVID (aka native VLAN ID), tagged networks and untagged networks (the native VLAN ID is untagged).
That's it.

Maybe you do more with your switches than I do...

If that Guest VLAN IP was the GW IP of OPN for that VLAN and the switch used it as well, then that network was in trouble.
I suspect it's possible the working devices had proper ARP tables (pointing to OPN) while the W10 machine was pointing to the switch.