I have been at this for two days now, I cannot get this vlan to work I have a dell R730 running OPNsense in a VM it has two ports dedicated to it one wan one lan WAN is RJ45 LAN is SFP+ twinaxe cable that connects to a cisco 3650 switch that port I have set to Trunk vlans 1 and 10. pots 5 on said switch has the same config for my u7 pro AP, Finally port 35 is set as access for vlan 10 which is my desktop that I am testing this issue with. Vlan 1 can ping vlan 10s gate way however anything connecting to my vlan 10 SSID via my AP wont get an IP nor will my desktop which as I mentioned is connected via an access port to vlan 10 please help my brain hurts. Thank you in advance.
Which firewall rules did you create for the new VLAN?
The pre generated, and access to the internet
For the new VLAN there are no pre-generated. Only for the default LAN interface. Whenever you create a new interface you also need to create firewall rules for this one.
Then what's the folder that says "pre-generated rules 18" that have DHCP and others in it?
What's a "folder"? Post a screenshot, please.
Here is a SS of my rules for vlan 10 with the generated rules folder extended and not extended
I don't see any manually added "allow" rule for anything. Which is of course necessary.
I have these manually made one
Looks ok to me. Time to login to OPNsense via SSH and watch with tcpdump what is actually happening on the wire.
I was looking at the DHCP logs and it was listening, I am SSHing in now, just "tcpdump" in shell im assuming?
I did "tcpdump -i vlan0.10" it is empty nothing going on I just connected my phone to the SSID that is set to tag vlan 10. and the switch is set to trunk vlan 1 and 10 on the AP port, and the firewall to switch port is set to do the same. No DHCP IP on my phone no tcpdump traffic on the vlan eitther
Then your AP is not forwarding the frames as you expect it to do.
I thought it was AP as well but if I set one of my switch ports to access vlan 10 that end device does not get connection to vlan 10 either. Im beginning to think it may be something with my switch it is a cisco 3650 if you know anything about them.
"switchport mode trunk" set on all ports carrying tagged VLANs?
1,10 on switch to firewall and AP to switch. then access 10 on the end device i am using for testing
What do you mean, "1,10"? Show the complete port config, please. Just the relevant one, of course.
Interface GigabitEthernet1/0/5 (AP)
Switchport allowed vlan 1,10
Switchport mode Trunk
!
Interface GigabitEthernet1/0/38 (end device)
Switchport access vlan 10
Switchport mode access
!
Interface TenGigabitEthernet1/1/3 (firewall)
Switchport allowed vlan 1,10
Switchport mode trunk
!
Is VLAN 1 tagged or untagged on the other devices? Also puzzled that it's not "switchport trunk allowed vlan ..."
You might want to try to remove that completely. It's not really necessary if you can trust the connected devices.
interface TenGigabitEthernet1/1/3 & 1/0/5
switchport trunk allowed vlan 1,10
switchport mode trunk
sorry I was away from my PC when I wrote the last response, this is the direct copy from my switch and 1 is untagged
So "switchport trunk native vlan" is not set at all and so has the default value of 1, I guess?
Well, since this is Cisco ... did you create the VLAN in the VLAN database? You can reference non-existing VLANs in the config and the switch will happily accept that but they won't work.
"show vlan brief"
1 default active Gi1/0/1, Gi1/0/2, Gi1/0/3, Gi1/0/4
Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9
Gi1/0/10, Gi1/0/11, Gi1/0/12
Gi1/0/13, Gi1/0/14, Gi1/0/15
Gi1/0/16, Gi1/0/17, Gi1/0/18
Gi1/0/19, Gi1/0/20, Gi1/0/21
Gi1/0/22, Gi1/0/23, Gi1/0/24
Gi1/0/25, Gi1/0/26, Gi1/0/27
Gi1/0/28, Gi1/0/29, Gi1/0/30
Gi1/0/31, Gi1/0/32, Gi1/0/33
Gi1/0/34, Gi1/0/35, Gi1/0/36
Gi1/0/37, Gi1/0/39, Gi1/0/40
Gi1/0/41, Gi1/0/42, Gi1/0/43
Gi1/0/44, Gi1/0/45, Gi1/0/46
Gi1/0/47, Gi1/0/48, Gi1/1/1
Gi1/1/2, Te1/1/4
10 IOT active Gi1/0/38
1 is native and I did give it an ip for ssh purposes. I just removed the trunk 1 on 1/0/5 and 1/1/3 still seems to be fucntion as normal with out trunking 1 on those interfaces but vlan 10 still no dice
I guess the AP is next ... I'm out of ideas.
Yeah, I am on the same page, but I have that port 38 and it is not getting communicate with the firewall either and the AP would not affect that client
Quote from: grant4790 on March 24, 2024, 08:48:52 PM
I thought it was AP as well but if I set one of my switch ports to access vlan 10 that end device does not get connection to vlan 10 either.
You should solve this issue first, if your switch doesn't communicate over L2 between two ports in the same VLAN, any uplink towards OPNsense (or the like) isn't going to work either.
Are you sure your AP is bridging the SSID assigned VLAN to the switchport, or is it actually routing (capture switchport traffic, see Cisco docs).
A few other observations:
* Your Trunk ports are missing encap config
switchport trunk encapsulation dot1q* OPNsense is using 802.1Q VLAN tags, don't use untagged VLAN's on Trunks
* If you _do_ need untagged VLAN's on Trunks (hint: you don't) assign any native VLAN to that port except for VLAN 1. Don't use VLAN1 in any VLAN design, just leave it as the DEFAULT which it is by default... (no pun intended)
* So instead of VLAN 1 & VLAN 10, use something like VLAN 10 & VLAN 11 (or any other id's below 4095). Of course both VLAN id's needs to be configured at OPNsense and assigned to your Trunk between switch and OPNsense
* If you're going to use multiple links between your switch and OPNsense configure a LACP Trunk. Otherwise you have to deal with Spanning Tree and other looping fun which introduces unneeded complexity. Test your topology first with a single interface before diving into multiple links (and LACP)
Quote from: netnut on March 25, 2024, 01:05:36 AM
* Your Trunk ports are missing encap config
switchport trunk encapsulation dot1q
* OPNsense is using 802.1Q VLAN tags, don't use untagged VLAN's on Trunks
OMG! I would definitely apply for the "resident Cisco wizard" role on this forum having run an ISP on Cisco gear for 25 years, but I'd never thought of a
current switch with anything but dot1q as the default.
As for the untagged/native VLAN issue - that's for the OP to fix afterwards, IMHO. While OPNsense and the underlying FreeBSD don't always work as expected with an untagged VLAN on a trunk port, that definitely should not keep a new VLAN from working at all.
Quote from: netnut on March 25, 2024, 01:05:36 AM
* Your Trunk ports are missing encap config
switchport trunk encapsulation dot1q
This command will not work per Interface on 3650 and other legacy switches. These command is for MLS switches.
3650 already by default supports 802.1Q and its the only encapsulation it supports by default.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3650/software/release/3e/vlan/configuration_guide/b_vlan_3e_3650_cg/b_vlan_3se_3650_cg_chapter_0100.html
Regards,
S.
Quote from: Seimus on March 25, 2024, 01:38:23 AM
Quote from: netnut on March 25, 2024, 01:05:36 AM
* Your Trunk ports are missing encap config
switchport trunk encapsulation dot1q
This command will not work on 3650 and other legacy switches. These command is for MLS switches.
3650 already by default supports 802.1Q and its the only encapsulation it supports by default.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3650/software/release/3e/vlan/configuration_guide/b_vlan_3e_3650_cg/b_vlan_3se_3650_cg_chapter_0100.html
Regards,
S.
I was about to say this as it indeed does do 802.1q by default and the encapsulation dot1q is an invalid command on my switch. If I untruck vlan 1 I get locked out of Opense managment ssh and webgui. still routes traffic though
Alright, @grant4790 - please show the output of `ifconfig` on your OPNsense.
ifconfig
enc0: flags=0<> metric 0 mtu 1536
groups: enc
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
syncpeer: 0.0.0.0 maxupd: 128 defer: off
syncok: 1
groups: pfsync
hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: WAN (wan)
options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
ether 14:18:77:62:dc:fe
inet xxx.xxx.xxx.xxx netmask 0xfffffc00 broadcast xxx.xxx.xxx.xxx media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
hn1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN (lan)
options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
ether 14:18:77:62:dc:ff
inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Vlan10 (opt2)
options=80000<LINKSTATE>
ether 14:18:77:62:dc:ff
inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
groups: vlan
vlan: 10 vlanproto: 802.1q vlanpcp: 0 parent interface: hn1
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wg1: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
description: wireguard (opt1)
options=80000<LINKSTATE>
inet 10.10.10.1 netmask 0xffffff00
groups: wg wireguard
nd6 options=9<PERFORMNUD,IFDISABLED>
root@OPNsense:~ #
And hn1 is the proper interface connected to your trunk port on the Cisco side?
interface TenGigabitEthernet1/1/3 & 1/0/5
switchport trunk allowed vlan 1,10
switchport mode trunk
I dont like this. You have Port set towards your AP as TRUNK. But does your AP know how to TAG?
In simple words >
access (switchport) port
a port that can be assigned to a single VLAN. The frames that arrive on an access port are assumed to be part of the access VLAN. This port type is configured on switch ports that are connected to devices with a normal network card, for example a host on a network.
trunk port
a port that is connected to another switch. This port type can carry traffic of multiple VLANs, thus allowing you to extend VLANs across your entire network. Frames are tagged by assigning a VLAN ID to each frame as they traverse between switches.
In simple term what this does mean is, that when on a switch you set the port in access, the Switches assume the ingress traffic that is coming is not TAGGed, so it will TAG it for you on and remove the TAG once again if the traffic within the same device comes back to that specific port or any other access port within the VLAN.
If its a Trunk port the Switch assumes the ingress traffic is already TAGGed with the specific VLAN allowed on the TRUNK.
VLANs are basically about logical segmentation and who strips/assigns the TAG. If its ACCESS Switch/GW will do it if its TRUNK its on the HOST/GW.
Regards,
S.
yes I have 2 10G ports the other is shut down so this is for sure it
yes my AP can tag it is a U7 pro with multiple SSIDs 3 for vlan 1 (just the different GHz) and 1 for vlan 10 IOT devices
As advised do not use VLAN1 for anything on CISCO switches. That VLAN is exclusive used by Cisco for their control plane on their Switches. It may still work but its advised not to use VLAN1.
If your host connected to YOUR AP over VLAN10 have issues to reach the GW. I would suspect misconfiguration on the Switch or AP side.
Try to set the switch port side do access vlan 10, and configure the AP without VLAN Encapsulation to see if it passes traffic thru VLAN 10 on switch towards the OPN.
Regards,
S.
okay I will work on taking VLAN 1 off, I have my phone and the switchport mode access on port 38 which is a windows machine they could ping each other from switch to AP
Laptop Switch AP
Vlan10 vlan10/P38 IOT network Port 5
As a few more people have jumped on this thread I want to summarize what we know so far. I have a VM running OpnSense on a Dell R730 native OS is windows server 19 with a 10g SFP+ connection to a Cisco 3650 SFP+ 10g connection that is set to trunk vlan 1 and 10, 1 is set to native and if I remove vlan 1 from the trunk I lose the opnsense gui and ssh for some reason. I have a windows end device connected to port 38 on that same switch configured as an access port for vlan 10, I have a u7 pro AP on port 5 of that switch configed as a trunk port for 1 and 10 as well on unifi controller I have two networks 1 for vlan 1 and one for vlan 10, 3 SSIDs for vlan 1 and 1 for vlan 10. I have connected my phone to vlan 10 and can ping said phone from the windows machine also on vlan 10. I do have the vlan set up in OPNsense but i can not get an IP from DHCP on either the wireless or wired devices.
Thank you all for the help so far I hope this summary helps clarify
Can you do,
While the current config the one with trunk on the ports >
show int Te[port towards OPN] trunk
show int Te[port towards PC] trunk
show int Te[port towards AP] trunk
And as well while a device is connected to your AP, do on teh switch
sh mac address-table interface Te[port towards AP]
Regards,
S.
Quote from: Seimus on March 25, 2024, 02:16:59 AM
Can you do,
While the current config the one with trunk on the ports >
show int Te[port towards OPN] trunk
show int Te[port towards PC] trunk
show int Te[port towards AP] trunk
And as well while a device is connected to your AP, do on teh switch
sh mac address-table interface Te[port towards AP]
Regards,
S.
show interfaces tenGigabitEthernet1/1/3 trunk
Port Mode Encapsulation Status Native vlan
Te1/1/3 on 802.1q trunking 1
Port Vlans allowed on trunk
Te1/1/3 1,10
Port Vlans allowed and active in management domain
Te1/1/3 1,10
Port Vlans in spanning tree forwarding state and not pruned
Te1/1/3 1,10
show interfaces GigabitEthernet1/0/5 trunk
Port Mode Encapsulation Status Native vlan
Gi1/0/5 on 802.1q trunking 1
Port Vlans allowed on trunk
Gi1/0/5 1,10
Port Vlans allowed and active in management domain
Gi1/0/5 1,10
Port Vlans in spanning tree forwarding state and not pruned
Gi1/0/5 1,10
show interfaces GigabitEthernet1/0/38 trunk
Port Mode Encapsulation Status Native vlan
Gi1/0/38 off 802.1q not-trunking 1
Port Vlans allowed on trunk
Gi1/0/38 10
Port Vlans allowed and active in management domain
Gi1/0/38 10
Port Vlans in spanning tree forwarding state and not pruned
Gi1/0/38 10
sh mac address-table interface gigabitEthernet1/0/5
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1 4abd.fb3e.9eac DYNAMIC Gi1/0/5
1 9c05.d643.c1b9 DYNAMIC Gi1/0/5
1 a64a.6488.9532 DYNAMIC Gi1/0/5
1 ae5a.8867.163d DYNAMIC Gi1/0/5
1 bad2.b99c.0125 DYNAMIC Gi1/0/5
10 ba41.46ea.f647 DYNAMIC Gi1/0/5
Total Mac Addresses for this criterion: 6
Thanks.
So looks like the switch allows the proper VLANs on the TRUNKs
Also if >
10 ba41.46ea.f647 DYNAMIC Gi1/0/5
Is your phone, that means switch sees this MAC being announced in VLAN10 thus the AP is Encapsulating it correctly.
DHCP server, is configured on OPN?
Can you show the configuration?
Regards,
S.
Here is an SS of the DHCP page for vlan10 on opnsense
Looks as well good to me.
One thing, so you can ping across same broadcast domain VLAN10 PC to VLAN10 AP Phone. When you statically assign the IPs to your devices correct?
Can you ping as well the GW on VLAN 10 from a device on VLAN10?
Regards,
S.
no that was over the auto back up assigned IPs APIPA i think is the protocol.
No ping of gate way from laptop with static and APIPA IPs
Well from APIPA I would not even expect to be able to ping.
But if you assigned static IPs from the VLAN10 pool and still can not ping this smells fishy.
Can you set that PC to static IP with IP from the Pool of VLAN 10 and perform continuous ping?
Set all your Rules on OPN, even the default deny to be logged. And have a look at live view. I know you mentioned capture didn't show any traffic, but try to have a look like this if you see something.
And as well if you can do >
sh mac address-table interface Te[port towards OPN]
Regards,
S.
show mac address-table interface tenGigabitEthernet1/1/3
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1 1418.7762.dcff DYNAMIC Te1/1/3
Here is the mac address table table for the interface facing opnsense, I have the windows machine ip to 192.168.10.101 and pinging 10.1 continuously.
could any of these settings be affecting it? or the fact that it is a hyper v vm? i mean that NIC port on the server is dedicated to just the VM and nothing else
Quote from: grant4790 on March 25, 2024, 03:21:55 AM
could any of these settings be affecting it? or the fact that it is a hyper v vm? i mean that NIC port on the server is dedicated to just the VM and nothing else
These should be at default in most case of the scenarios. No need to play with them
Quote from: grant4790 on March 25, 2024, 02:48:20 AM
show mac address-table interface tenGigabitEthernet1/1/3
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1 1418.7762.dcff DYNAMIC Te1/1/3
Here is the mac address table table for the interface facing opnsense, I have the windows machine ip to 192.168.10.101 and pinging 10.1 continuously.
Well here we have the problem.
As you can see your MAC (VLAN inherits its MAC from parent) 14:18:77:62:dc:ff is seen only advertised under VLAN1 but not under VLAN10.
You should see here an entry for 14:18:77:62:dc:ff in VLAN10 as well. Thats the reason you can not ping, or reach at all the OPN GW, as from perspective of the Switch no device on VLAN10 is seen on the TRUNK port.
Regards,
S.
Great we found the issue thank you for the help! But how do I fix it?
Sadly I do not run my OPN in Virtual setup, only as baremetal.
You will need to dig more into the VM configuration probably, or hopefully somebody who uses OPN as VM can help.
From network perspective aka Switch side, as long you have the port configured as TRUNK and Allow proper VLANs it should be OK.
Regards,
S.
Maybe this will help?
https://www.youtube.com/watch?v=M8PLt5-dmoA
Also check the Hyper V documentation for VLAN and Trunking.
Regards,
S.
Quote from: Seimus on March 25, 2024, 01:38:23 AM
Quote from: netnut on March 25, 2024, 01:05:36 AM
* Your Trunk ports are missing encap config
switchport trunk encapsulation dot1q
This command will not work per Interface on 3650 and other legacy switches. These command is for MLS switches.
3650 already by default supports 802.1Q and its the only encapsulation it supports by default.
I stand corrected. Last Cisco switch I touched was a 3750 a few decades ago ;), "switched" to another platform and never looked back...
I see the OP is in good hands with you!
Thank you all for all your help! I am going to try that video I just watched it and it looks promising!! Thank you guys so much I will update you all when I get a chance!
Quote from: netnut on March 25, 2024, 08:33:11 PM
I stand corrected. Last Cisco switch I touched was a 3750 a few decades ago ;), "switched" to another platform and never looked back...
I see the OP is in good hands with you!
3750 was a good switch, basically a simple MLS, it was one of the first switches to support as well IPbase images and routing with possibility of multiple L2 encapsulations.
Quote from: netnut on March 25, 2024, 08:33:11 PM
I see the OP is in good hands with you!
Well just a casual network guy here :)
Quote from: grant4790 on March 25, 2024, 09:26:25 PM
Thank you all for all your help! I am going to try that video I just watched it and it looks promising!! Thank you guys so much I will update you all when I get a chance!
For sure let us know. Cause it seems you maybe are missing proper vSwitch configuration on the Hyper V.
Regards,
S.
UPDATE: yes the video worked that was indeed the issue!!! however I have a new one now...
I messed up and changed the listening interface for the webgui I still have SSH however it seems like everything I change on there does not fix my issue does anyone know how to change the listening interface via SSH
Glad it helped you so indeed you had miss-configured Hyper V.
In regards of Webgui access, ah those sweet beginner mistakes. Try this
https://docs.opnsense.org/troubleshooting/webgui.html
Otherwise you can rollback config to the last working one via ssh.
Regards,
S.
gorgeous thank you! worked like a charm!
Glad we were able to fix all your problems. If all is fixed please update your thread name with [SOLVED], lets keep this forum clean!
P.S. Always consult documentation, and check your MAC address tables on Switches and ARP tables on routers ;) > Networking 101
P.P.S Migrate your NOT tagged OPNGW to a proper VLAN TAGGED GW so you can get rid of that VLAN1 and let it be. You have a nice switch capable to do VLANs so why not to use it.
Regards,
S.