Messages

I still suspect it is something to do with your VLAN configuration and using VLAN1/LAN interface subnet. I have a similar configuration with Unifi APs and switches but I don't use VLAN1. My wired and wireless clients all connect to VLAN10, 20, 30 etc., and nothing connects to the LAN interface's subnet except for brand new trunk devices that need to be configured. My LAN interface subnet does not have a corresponding wireless network and I have a management VLAN that I use for administration of the Unifi devices.

Interesting approach.. I'll make note of your configuration approach as i work through all of this.

Have you tried looking at Firewall->Log Files->Live View and watching for traffic from a guest network device trying to access the Internet? You can set the source filter to the IP of the network device. If you don't see traffic there, it's not a firewall/rules problem.

I have not - I'll investigate soon, but leaving for a trip, and wont be back for some time.

How did you confirm your DNS service is "reachable" on VLAN 20? Did you set a PVID on your switch for port 1 and 4 (I'm not sure how your switch handles native traffic)?

For simplicity sake - I disabled the captive portal option in Unifi Console, and Clients are able to connect to the Guest Wifi - get an IP and connect to the internet - which makes me suspect DNS is reachable heh. 

I'm told that my PVID settings are correct. Given that my Access Point (AP) on port 4 should be broadcasting SSIDs for both the primary LAN (likely untagged, VLAN 1) and the guest network (VLAN 20), PVID of 1 for Port 4 is correct if I want the untagged traffic from the AP to be associated with VLAN 1.

And since Accepted Frame Type is set to All, the switch is accepting both untagged (for the primary LAN) and tagged traffic (for VLAN 20).

[Suggested Approach]

I am not familiar with your switch so I don't know how it handles tagged and untagged networks on the same port. Can you please try taking ports 1 and 4 off of VLAN 1 (remove them from untagged or put them as tagged on vlan1)?

When I marked ports 1 and 4 as tagged for VLAN1 and VLAN20 - all devices connected to the AP on port 4 lost internet - i reverted it back to VLAN1 all ports as untagged.

Suggested Approach
Port 1 (Connected to Firewall):
VLAN 1: Should remain untagged because this is typically the default/native VLAN, and most devices expect the default VLAN to be untagged.
VLAN 20: Should be tagged so that the firewall can send out tagged traffic for the guest network.

Port 4 (Connected to AP):
VLAN 1: Should remain untagged if the AP or devices on VLAN 1 expect untagged traffic.
VLAN 20: Should be tagged so that the AP can properly segregate the traffic and route VLAN 20 traffic to the appropriate SSID or port.

When I tagged VLAN 1 on port 4, all the traffic that used to be untagged (and thus understood as part of VLAN 1) became tagged. So any devices on VLAN 1, expecting untagged traffic, could no longer correctly process the traffic. Hence why they lost internet.

Can your NAS or NUC access the WAN? I assume your NAS and NUC are on VLAN1.

Yes - All other Main LAN devices (VLAN1) Wifi/Wired are functioning as expected without issue.

LAN - 10.0.1.0/24 Subnet
VLAN20 - 10.0.20.0/24 Subnet

LAN Firewall Rules

Destination needs to be the unifi controller, not "This firewall".

The Unifi console software is running on the firewall (via plugin - from mimugmail repo) - there is no seperate piece of hardware.

[NEW PROBLEM]

Alright ended up with a NICGIGA Managed 8 Switch (S25-0801-M), and figured out how my ports should be tagged



Guests are able to connect and get an IP Address.

NEW PROBLEM
I'm running UniFi Network Application 8.2.93 on my OPNsense firewall and trying to use UniFi's built-in captive portal instead of OPNsense's. Clients on the guest network (VLAN 20) are getting IP addresses and correct gateway/DNS info (10.0.20.1), but they can't access the internet or see the captive portal.

VLAN 20 is properly configured on the firewall, switch, and AP. The DHCP server is working fine. I've verified that UniFi's captive portal uses ports 8880 and 8843. I'm not sure if my firewall is allowing traffic to the necessary ports. DNS is properly configured and reachable. I temporarily disabled block rules, but the issue persists.

What might I be missing?

[Update:]

You should be able to ssh into your UniFi APs and run something like tcpdump -nnei eth0 vlan to see if you see any tagged frames, and do the same on your opnsense box (except igc1 instead of eth0). If you see tagged frames leaving one and not arriving on the other, it's likely that the switch is eating them. You could also try filtering by MAC address (of a WiFi client).

heres the result on my firewall

Code Select Expand

root@OPNsense:~ # tcpdump -nnei igc1 vlan
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igc1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
0 packets captured
26878 packets received by filter
0 packets dropped by kernel

results from unifi AP in my bedroom connected to the switch

Code Select Expand

U6E-Room-BZ.6.6.73# tcpdump -nnei eth0 vlan
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
0 packets captured
0 packets received by filter
0 packets dropped by kernel
6 packets dropped by interface

The results suggest the UniFi AP isn't capturing any VLAN-tagged packets on its eth0 interface

I'll try connecting the UniFi AP in my bedroom directly to the OPNsense firewall, bypassing the switch, to see if VLAN-tagged packets start appearing. If they do, the switch might be the issue.

Update: Confirmed, the main switch is the issue - i directly connected my bedrooms AP to the Firewall, and the Guest Network worked immediately - Got an IP and everything... Fantastic

For anyone thats curious my current switch is a TRENDnet TEG-S380 (Version v1.xR). Gonna try a TP-Link TL-SG108-M2 - as there are reports of people not having issues with that switch and passing VLAN tagged traffic... Will report back once I receive it

[Can your MoCA devices bridge 802.1q VLAN tagged packets?]

You'd be correct that my diagram is incorrect the MoCA adapter is connected to the 8 port switch. Good catch.. not sure that changes much

Well if the diagram was correct, I'd want to know how you got both the switch and the MoCA adapter connected to igc1 at the same time ;D

... but seriously, I think the important point is that the 8-port switch is the common element in the path between the fireall and all of the APs, and MoCA stuff isn't (assuming the problem is manifesting on the "Upstairs (My Bedroom)" AP as well as the others).

Good idea - i could tell my Unifi Console to pass the Guest Network SSID onto the AP in my bedroom only, and then another AP and then compare to see if the VLAN Tags are terminating after the main switch.

I'll try your suggestion to SSH into the UniFi APs and use tcpdump -nnei eth0 vlan to check for tagged frames. I'll do the same on the OPNsense box (igc1) and see if there's any difference between the frames being sent and received.

If I find that the frames are being dropped by the switch, I might look into replacing it with a managed one

Regarding to MoCA Adapters... According to goCoax's FAQs

Can your MoCA devices bridge 802.1q VLAN tagged packets?
Yes, some MoCA devices can bridge 802.1q VLAN tagged packets. However, it is important to check the specifications of the specific MoCA device you are using to ensure that it supports VLAN tagging. Some MoCA devices may not support VLAN tagging, or may require specific configuration settings to enable this feature.

That said, it *may* be that the 8-port switch in the OP's diagram is not passing the tagged frames. I assume that there's an error in the network diagram, and the MoCA adapter is connected to that switch, and not to the firewall directly. OR there could be something else going on.

You'd be correct that my diagram is incorrect the MoCA adapter is connected to the 8 port switch. Good catch.. not sure that changes much

I'm having trouble setting up a guest network on my OPNsense firewall, which is also running the UniFi console software. My goal is to use UniFi access points (APs) without needing any additional UniFi hardware. However, I'm struggling with VLANs, which seems to be the main issue.

The WiFi network on my main LAN is working perfectly, so the APs are functioning as expected.

Here's a quick overview of my setup:

OPNsense firewall connected to an unmanaged switch.
That switch is connected to other unmanaged switches, which then connect to three UniFi APs.
My basic understanding is that I need to configure a VLAN in OPNsense for the guest network. The APs should then pick up this VLAN and broadcast an SSID associated with it. I've followed some initial steps:

Created a VLAN (vlan01 with tag 20) and assigned it to an interface (Guest_VLAN).
Enabled DHCP on the Guest_VLAN interface.
Verified that the VLAN is properly tagged on the interface connected to my APs.
However, my clients still aren't receiving IP addresses when they connect to the guest network. I suspect this might be due to my limited understanding of VLANs, or perhaps something's missing in my configuration. Since I'm using unmanaged switches, I'm not sure if this setup is correct, and I would greatly appreciate any step-by-step guidance to get this working properly.

System Information
OPNsense 24.7.1-amd64
FreeBSD 14.1-RELEASE-p3
OpenSSL 3.0.14

CPU
Intel(R) Pentium(R) Silver N6005 @ 2.00GHz (4 cores, 4 threads)















Updated Diagram (8/10)

Forgive me as im a complete novice when it comes to this sort of thing, but i need assistance in figuring out why my system log is flooded with a level 7 log level kernel notice relating to traffic that is unable to be forwarded, clearly relating to ipv6 (which i know very little about).

My log has been flooded for so long, ive reached 51GB worth of logs AND climbing!

The log message indicates that the system is unable to forward traffic from one interface (igc1) to another (igc0) and specifically mentions that it's having trouble with IPv6 traffic using both UDP (nxt 17) and a non-UDP (nxt 58) protocol.

I'm honestly not really sure where to start with troubleshooting on this one, but i will say that there has been strange network behavior with some of my TP-Link smartplugs (no longer accessible/toggleable in the Kasa app, or Sense Home App).

snippet of log

Code Select Expand

2023-10-02T12:16:41-04:00 Notice kernel <7>cannot forward src fe80:2::3ff0:fb6b:56af:56d7, dst 2001:4860:4860::8888, nxt 17, rcvif igc1, outif igc0
2023-10-02T12:16:13-04:00 Notice kernel <7>cannot forward src fe80:2::3ff0:fb6b:56af:56d7, dst 2001:4860:4860::8888, nxt 58, rcvif igc1, outif igc0
2023-10-02T12:16:05-04:00 Notice kernel <7>cannot forward src fe80:2::a4f2:2ccb:4b03:18b2, dst 2001:4860:4860::8888, nxt 58, rcvif igc1, outif igc0
2023-10-02T12:15:57-04:00 Notice kernel <7>cannot forward src fe80:2::9ecc:6e40:bf07:3a9, dst 2001:4860:4860::8888, nxt 58, rcvif igc1, outif igc0
2023-10-02T12:15:12-04:00 Notice kernel <7>cannot forward src fe80:2::3ff0:fb6b:56af:56d7, dst 2001:4860:4860::8888, nxt 17, rcvif igc1, outif igc0
2023-10-02T12:15:05-04:00 Notice kernel <7>cannot forward src fe80:2::a4f2:2ccb:4b03:18b2, dst 2001:4860:4860::8888, nxt 58, rcvif igc1, outif igc0

I wonder if an errant setting was enabled or setting that might be causing this.. but not sure which.

Things to know - i have Verizon Fios. my igc0 interface is my WAN interface. igc1 is my LAN interface.
under Interfaces->WAN IPv6 configuration is set to DHCPv6
under Interfaces->LAN IPv6 configuration is set to Track Interface
under Interfaces->Settings IPv6 DHCP Prevent Release is enabled (recently enabled as of today, to see if this will help)

Running
Versions   OPNsense 23.7.5-amd64
FreeBSD 13.2-RELEASE-p3
OpenSSL 1.1.1w 11 Sep 2023

@mimugmail - with the release of 23.7, are there plans now to update to the 7.4 branch? i dont quite understand what you meant by there needing to be a manual recovery for mongodb since the 7.4 branch requires 4.0

i guess has things settled a bit yet? hehe

I've had this persistant issue with the os-unifi7-maxit plugin, where if the firewall experiences a non-clean shutdown, the unifi plugin is unable to recover - (i.e: unable to start back up). And everytime this happens i have to not only uninstall the plugin, but go into shell, and delete the entire /usr/local/share/java/unifi/ folder, before reinstalling it.

After a non-clean shutdown, my system log is flooded with the following notice/errors
2023-07-03T13:28:18-04:00   Notice   kernel   <6>pid 88314 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:28:14-04:00   Notice   kernel   <6>pid 69372 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:28:09-04:00   Notice   kernel   <6>pid 68133 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:28:05-04:00   Notice   kernel   <6>pid 66300 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:28:00-04:00   Notice   kernel   <6>pid 57931 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:27:55-04:00   Notice   kernel   <6>pid 56444 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:27:51-04:00   Notice   kernel   <6>pid 55303 (mongod), jid 0, uid 975: exited on signal 6   
2023-07-03T13:27:46-04:00   Notice   kernel   <6>pid 54073 (mongod), jid 0, uid 975: exited on signal 6

It appears to be something to do with mongodb, corruption of the DB? Pastebin of the last mongodb log.
https://pastebin.com/ze8ug4AD

i saw somewhere that "mongod --repair should be able to fix these kind of errors, but only if the fixes have been installed. According to the repair documentation, that only applies to Mongo DB 4.0.3 and later." - i see we are on mongodb 3.6.23  :-\

@mimugmail: What are the plans to move up to the latest controller version to 7.4.162? As well as incorporating a mongodb repair script?

[Update:]

Is it possible to utilize OPNsense to filter specific websites and redirect them to an existing proxy service (privoxy) that is active on another device on the same subnet? (As opposed to configuring each client's proxy settings with the privoxy server, and pushing through all web traffic through the proxy that is using a VPN).

Or should i think about filtering specific URLs through a VPN? Is that even possible?

[the idea being, that any device on the LAN, can go to said URL, and it would be tunneled through the proxy, without any additional configuration on the client side]

Why?: My ISP decided this URL should be blocked (ip aware blocking, dns changes made no difference), and thus a proxy/vpn was necessary to overcome that.

Just a thought i had, and am curious if either scenario is feasible or not. And if so, how would one get that configured?

Update: This looks like what im looking for https://docs.opnsense.org/manual/how-tos/proxytransparent.html

Update 2: I think im looking at this the wrong way, maybe i can utilize HAProxy... i might have to post this on the HAProxy forum though...

[But if you would like to do it my way then you will need to create a virtual IP that is in a different subnet than any of your other networks. Preferably you would chose an IP that belongs to the localhost subnet in order to avoid IP conflicts in your local network.]

apologies, my tired eyes at 4am missed this part

• 
  But if you would like to do it my way then you will need to create a virtual IP that is in a different subnet than any of your other networks. Preferably you would chose an IP that belongs to the localhost subnet in order to avoid IP conflicts in your local network.

i also misunderstand that setting a virtual IP that has a "loopback" does not automatically mean that it serves to "Loopback" to the firewall. that one was on me.

I updated the virtual ip to be 127.4.4.3 (mirroring your setup)

and updated SSL_Server to 127.4.4.3

HTTPS and HTTP Front Ends to listen on 127.4.4.3.

Updated unbound overide IP value to 10.0.1.1 (Firewall/OPNSense IP)

Restarted HAProxy - and its still not working. i wonder what i did wrong...

Update: Rebooted the firewall and that fixed it...

[Host Overrides]

Im up and running (Everything works!) I Figured out my Cloudflare DNS challenge issue (you have to make sure to get the Token that is displayed AFTER you create a Cloudflare API Token for the Zone.DNS read and Zone.DNS Edit... I mistakenly thought it was the Global ID Key..)

Now the only thing i cant get to work is the Unbound DNS Override.

You stated

If you are running all of your services on your 1st level subdomain "your_subdomain.dedyn.io" than you will just need to override this one.

Since im utilizing a wildcard, i figured it should work this way, so that any subdomain i enter, will be redirected to HAProxy's SNI_Frontend. And since its listening on 0.0.0.0, i figured the virtual IP should work - i also tried the Firewalls IP address with no luck.



And yes - the virtual ip is set to loopback.

Also - the Unbound DNS Overrides section looks different, now theres 2 tabs (Host Overrides), a main entry and an aliases entry,

then a (Domain Overrides) tab.