I Set Up A VLAN But Can't Ping Systems On It

[isaacthekind]

I currently have 3 interfaces: LAN, WAN, and DEVICES.

DEVICES is a VLAN assigned to LAN. Both have DHCP enabled, and their subnets are:

LAN 10.0.0.0 DHCP range 10.0.0.50 - 10.0.0.254
DEVICES 10.0.1.0 DHCP range 10.0.1.50 - 10.0.1.254

I can't ping systems on DEVICES from LAN even after adding rules on DEVICES to allow any protocol from LAN net to DEVICES net, and I actually can't even ping systems on devices from OPNsense itself.

I'm not sure what I'm missing here. Help would be appreciated, thanks for taking the time to read this.

EDIT: I should also have mentioned I do have internet on systems on DEVICES, for example pinging google from a system on DEVICES succeeds.

[netnut]

I currently have 3 interfaces: LAN, WAN, and DEVICES.
DEVICES is a VLAN assigned to LAN. Both have DHCP enabled, and their subnets are:

Don't mix LAN (raw) and DEVICES (vlan) with a single interface. Use the current LAN physical interface as VLAN parent (trunk), and configure two VLANs (LAN and DEVICES).
Be aware that changing your LAN interface might disconnect you from CLI/GUI, so be sure you prepare your change smart. Configure your downstream switch with both VLAN id's (tagged)

Or start clean, during the setup of OPNsense you've been asked to create VLAN interfaces, do that at this point (create LAN & DEVICES VLAN) and finish setup as usual.

I can't ping systems on DEVICES from LAN even after adding rules on DEVICES to allow any protocol from LAN net to DEVICES net, and I actually can't even ping systems on devices from OPNsense itself.

If you want a packet going out of a network segment (LAN) to another network segment (DEVICES), you should configure your rule at the LAN interface. From a firewall point of view a packet is coming INto the LAN firewall interface from LAN network, the rest is covered by stateful filtering ;-).

[isaacthekind]

Ok, welp I failed that pretty badly. Just managed to lock myself out of my GUI and TUI for a long time and spent way too long trying to enter the long password.

I'm trying the approach you suggested but still having trouble. I've configured 2 VLANs CORE and DEVICES, both on bce1. And enabled DHCP on both of them. I also went into my switch and tagged port 3 with vlan 2. But still when I plug a device, like my server into port 3, it isn't being assigned an IP on the DEVICES VLAN associated with port 3.

[gcorre]

Hi!

Can you show how you configured your DHCP?

Also, did you configured the port where you plug your device as an access port for vlan 2?

It should not be a firewall rule issue as OPNsense add automaticaly an "allow" DHCP rule when you activate DHCP on an interface.

You can check if your firewall see your device DHCP request by doing packet capture here : "Interfaces / Diagnostics / Packet capture".
I suggest you select all interfaces to be sure that you are not missing any traffic.
It's also possible to look at DHCP logs here : "Services / DHCP / Log File".

You can see my DHCP configuration in attachment.

Cheers

[netnut]

Ok, welp I failed that pretty badly. Just managed to lock myself out of my GUI and TUI for a long time and spent way too long trying to enter the long password.

Step by Step, don't try to do everything at once....

First build your topology, check, check, double check, next are things like DHCP, DNS, etc. So configure your VLANs at OPNSense, configure your switch uplink port (trunk, all vlans) other ports as "access" with desired VLAN and assign IP configuration to OPNsense interfaces and hosts connected to switch.

WAN --- [OPNSENSE] --- BCE1 (VLAN TRUNK)
                                       |
                                       |
                                       |
            VLAN 2 (CORE) / VLAN 3 (DEVICES) / VLAN X (X)
                                       |
                                       |
                                       |
                                [SWITCH]
                 Uplink --->      |--- Port 1 (VLAN 2+3+X TRUNK)
                                       |
                                       |--- Port 2 (VLAN 2 ACCESS)
                                       |
                                       |--- Port 3 (VLAN 3 ACCESS)
                                       |
                                       |--- Port 4 (VLAN X ACCESS)

If you accomplished this step open up your firewall with an allow any any (you're still building your network, fine tuning is done when the fundamentals are right).

Create a firewall rule at both/all VLAN interfaces, like:

Action: Pass
Interface: VLAN?
Direction: In
TCP/IP Version: IP4+IP6
Protocol: any
Source: any
Destination: any

Now connect hosts to your switch access ports one in VLAN 2 and one in VLAN 3, you don't care about DHCP yet, so configure a static IP in the subnet of your choice as configured at OPNsense. Try to ping or whatever between these hosts, if that doesn't work you did something wrong, time to troubleshoot.

If this all works you can enable DHCP, there isn't much to configure, DHCP is made death simple in OPNSense, just assign a pool to the corresponding VLAN interfaces and you're done. Again, step by step....

Ow, and don't use VLAN1 (Default VLAN) in a VLAN design, leave as is and use something between 2 and 4095. VLAN 1 will work and has nothing to do with your "challenge", but at this stage it's a perfect time to ditch VLAN1 from your topology.
Also stay away from "native vlan ids" on Trunk ports (unless you absolutely know what you're doing), so just use tagged VLANs on trunk ports and a single vlan with an access port.

[isaacthekind]

i got a new device to run the firewall on (Protectli Vault FW4B), so it took me a while to configure everything and get back to where i was. Some of the names are slightly different, however I've found myself stuck on the exact same issue.

gcorre, yes I can show my VLAN DHCP configuration, a screenshot is linked.

netnut, thanks for providing so much detail. I did my best to follow these steps to a tee, but I'm still having the same problem. I'll include a detailed diagram of my current topology. You'll notice there is one blank bubble in the photo, this is the one I have to plug into to actually get any internet on for my desktop. I'll also include a photo from the switch GUI, but just FYI to obtain that photo I have to plug my desktop into port 4, so If you're wondering why port 4 rather than 2 is active in the screenshot, that is why, more importantly though, the screenshot shows the access VLAN tags. I've also configured firewall rules as suggested on CORE and DEVICES. When I connect, I get nothing, I can't ping any device and IP neigh shows no devices, even after reboot of all systems and the OPNsense router. I can however ping 10.0.2.1 and 10.0.3.1 from OPNsense, or from desktop if plugged into port 4.

[isaacthekind]

Also in the case that you want to diagram anything again, feel free to modify my diagram rather than having to make one from scratch. I'll include the Excalidraw file, you can open it on https://excalidraw.com/ .

[gcorre]

Can you show us a screenshot of the configuration of your VLAN CORE for example in :

• Interfaces / vlan core (or whatever you named it)"
• Interfaces / Assignment / vlan core
• Interfaces / Other Types / vlan core / edit (the small pencil)

[netnut]

First, your DHCP config is perfect, nothing to change there...  I guess you still have VLAN issues, which is understandable (you'll learn along the way), I suggested the following:

i got a new device to run the firewall on (Protectli Vault FW4B)

Nice, excellent device to learn and play with. Because you have a LAN and two OPT ports I suggest you take a slightly different approach than I suggested first. You can get this working with a single LAN interface (as you tried), but the following is more fail safe setup and gives you all the opportunities to play with VLANs and don't lock yourself out.

Configure your device as usual with a regular WAN and LAN interface, this LAN interface will be your "management" network. Now do all your VLAN magic at OPT1 or OPT2 (if you really on fire you can later configure a redundant LACP LAG port to your Cisco, but don't get to excited yet, first thing first...)

So do what you already did (assuming igb0 is WAN, igb1 is LAN, igb2 is OPT1 and igb3 is OPT2), but instead of using the LAN interface (igb1) use your OPT1 or OPT2 to connect your Cisco.
DONT CONFIGURE OPT1 or OPT2 with IP addess information, just assign the VLAN interfaces to it. You can now always use your LAN port to connect to OPNSense and go bezerk on the config of OPT1 /OPT2, if you make a mistake you always have your LAN interface to troubleshoot.

So only configure your CORE and DEVICES VLAN (with OPT1 or OPT2 as parent) and CONFIGURE those two VLAN interfaces with a gateway address at your OPNsense box, I guess you want 10.0.2.254/24 for CORE and 10.0.3.254/24 for DEVICES.

If your finished and still have problems, dump the config of your Cisco switch (please edit any secrets, passwords etc, don't need to share these)

[isaacthekind]

cgorre, yes, they are linked.

netnut, could you clarify what you mean here:

"So only configure your CORE and DEVICES VLAN (with OPT1 or OPT2 as parent) and CONFIGURE those two VLAN interfaces with a gateway address at your OPNsense box, I guess you want 10.0.2.254/24 for CORE and 10.0.3.254/24 for DEVICES."

I've set up TRUNK on OPT1, and made CORE and DEVICES vlans which have TRUNK as the parent, and I've got LAN set up with no children VLANs as a management network that i can fall back to but which I do not plug in when trying to get the VLANs working. Where is the gateway setting though? If you mean the setting Services > DHCPv4 > CORE > Gateway that becomes visible in DHCPv4 for CORE after assigning a static IP to CORE, I get a problem when I do that where it says that CORE 10.0.2.1 is not on the same subnet as the gateway 10.0.0.1. So I'm not really sure how to achieve this step. Could you clarity what IPs you think LAN, TRUNK, CORE, and DEVICES should have, which should have DHCP enabled, and what you mean when you say to assign the devices on these VLANs IPs without using DHCP? It seems to me that I have to enable DHCP to use the static mapping feature in the bottom of Services > DHCPv4 > INTERFACE NAME. I fully understand the topology you describe on the physical layer, everything is plugged in where it should be, but I'm getting lost on the network layer, as in which IPs go where.

Sorry for all the questions, I feel a bit silly getting so stuck with this.

EDIT: Please note that the screenshots were taken for gcorre with the network in the state it was in when they asked the question, they do not reflect any changes made for netnut.

EDIT 2: I don't understand what it would mean to have a "gateway address at my opensense box" if not that the gateway address is the adderss of LAN or TRUNK. But I'm not supposed to assign an IP to TRUNK or use LAN.

[isaacthekind]

Ok, I managed to get it to work. The issue was actually not OPNsense at all, but instead the switch. I guess Cisco has these things called smartport types, and I had to assign the appropriate type to the port which connects to the router. Once I assigned that port the "cisco router" type, everything worked. I had assumed that they were just labels, and didn't have any function beyond that.

I'll now take a look at the redundant LACP LAG port stuff that netnut mentioned.

Thank you both for all the help, really appreciated.

[netnut]

"So only configure your CORE and DEVICES VLAN (with OPT1 or OPT2 as parent) and CONFIGURE those two VLAN interfaces with a gateway address at your OPNsense box, I guess you want 10.0.2.254/24 for CORE and 10.0.3.254/24 for DEVICES."

Hmmm, might have rephrased that one ;-). What I meant to say is that your OPNsense VLAN interfaces are gateways for the specific VLAN they're connected to, but...

Ok, I managed to get it to work.

You did it 

I'll now take a look at the redundant LACP LAG port stuff that netnut mentioned.

Remember that you can have a LACP LAGG with a single port, so use your free OPT2 for that. If you manage to get it work, you can migrate your existing VLAN's to this trunk and add (you're now free OPT1) the other interface to it to make it redundant.

[isaacthekind]

I have one more question. It's still roughly on this topic, but maybe I should open a new thread. If that's the case just let me know.

So I have my VLANs all up and working. And i can get the rules on them to work the way i want. For ex being able to ssh into or ping other VLANs from CORE (at least when I turn on the rule to do so), but not the other way around. However I'm getting one weird behavior that is hard to understand. I'll include my current topology. Currently my goal is to get OpenWRT to broadcast 2 wireless VLANs: HOME and GUEST. I've plugged my wireless access point directly into igb2 to remove any complications that may arise as a result of going through the switch, though in time the goal is to plug it into the switch and free up igb2 for the LACP LAGG. I can ping the access point from OPNsense, but not from CORE. Even with an allow all rule on core and on WIRELESS. I'm really not sure how to debug this. I looked in the live logs and when I ping from desktop I can see it passing (picture included).

[netnut]

I can ping the access point from OPNsense, but not from CORE.

With your access point connected to igb_2 on OPNsense, the IP address of that interface is the gateway for your access point. Did you configure this IP address as gateway into your Access Point ? Depending on type/version/ap of OpenWRT it's something like this:

Code: [Select]

config interface '?'
        ...
option gateway '1.2.3.4' # <--- OPNsense interface IP of igb_2
        ...

Rember that the default logic of a starting config with OpenWRT is to have a LAN/WAN with NAT enabled. But it depends on the AP type, amount of ports, switch yes/no etc. So it's possible if you use this default config you trying to ping the OpenWRT WAN interface which has a firewall enabled. To dubble check if your topology is right, connect another device to igb_2 and configure it just like your AP and see if ping works. If it does, your topology is right but need to give OpenWRT some attention.

You probably like to read the "Dumb Access Point" guide at OpenWRT, this should be the OpenWRT way of doing things you want. When you connect the AP to your switch, you're capable of (don't need to if you don't want) to bridge your existing VLAN's behind one or more SSID'

It might help if you also add the network addresses in your diagram below the names, this helps to understand what you did build. Something like "CORE : 10.1.0.0/24", "DEVICES : 10.2.0.0/24", etc

[isaacthekind]

I've read the dumb access point guide before, and successfully implemented it which involves disabling DHCP, firewall, etc. But this was without trying to broadcast 2 separate networks and without having VLANs on OPNsense. Maybe I should just plug the access point into the switch since my whole reason for using igb2 was to avoid trouble and that seems not to have happened. I've included the updated topology image you suggested. I'm a bit worried about an X/Y problem here though, so maybe I can just tell you what it is I want to do and if you feel so inclined you can tell me how to approach it (hopefully I'm not being too demanding):

My goal is to have 2 networks broadcast from OpenWRT: HOME and GUEST. I want to configure HOME and GUEST through OPNsense and simply have OpenWRT be a dummy AP, I don't want to be managing separate VLANs on OpenWRT that are not visible in OPNsense. I'm unclear what subnet the cable I pass from the router to OpenWRT should be on, and what IP it should have. TRUNK maybe? A new interface with 2 VLANs on it an no IP kinda like a second TRUNK? I used a dashed line to indicate the part of the topology I don't know how to configure.