Messages

1

24.7 Production Series / VLAN traffic blocked on LAN interface

« on: September 08, 2024, 12:54:05 am »

I have a routed switch with 5 VLANs configured on it. I have a route and gateway configured on the OPNsense box. The firewall is blocking traffic in both directions on the 192.x network and the 10.x networks.

Code: [Select]

LAN 2024-09-05T17:06:36 10.0.1.100:36436 52.16.96.58:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.253:60322 162.159.140.167:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.253:59754 162.159.140.167:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:38264 54.73.190.247:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:35326 8.8.8.8:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:35326 8.8.8.8:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:35 10.0.1.248:55142 8.8.4.4:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:33 10.0.1.253:56942 172.66.0.165:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:32 10.0.1.253:56934 172.66.0.165:443 tcp Default deny / state violation rule
I tried an any any rule on the LAN interface, but it's still blocking traffic. Do I have to create a rule for each port? If so how would I handle services using random ports on the the other subnets?

Thanks again for all your help getting this far with my setup. Once I get this working I will tackle the final hurdle, multi-wan with traffic pinning =D.

2

24.7 Production Series / Re: Home use "easy button" question

« on: September 06, 2024, 06:26:25 pm »

I think I will mark this one resolved and start a new thread with the config challenges I have now. I have a couple experiments with rules going so hopefully one will work.

-Tory

3

24.7 Production Series / Re: Home use "easy button" question

« on: September 05, 2024, 07:12:55 pm »

I forgot to say THANK YOU, to all the folks who offered assistance! Gratifying to get everything working this far =D. Thank you!!

4

24.7 Production Series / Re: Home use "easy button" question

« on: September 05, 2024, 07:12:06 pm »

Ok, so it took me a while to get everything rejiggered. I reinstalled from scratch, installed my monitoring plugins, setup the KEA DHCP server, setup a "LAN" gateway for the firewall itself and applied all that and low and behold everything flipping works! I mean smoothly, I have 2 port forwards to the same subnet as the LAN interface working like a charm.

Plex is working using the private domain in Unbound, and a simple NAT rule. All my other services on the LAN subnet are working as expected.

Now to the current issue and what my next steps should be.

Traffic is being blocked from my VLANs unless I create an explicit firewall rule to allow it. I created an any any rule for the LAN but that didn't work for any of the subnets behind my local routed switch. See logs below:

Code: [Select]

LAN 2024-09-05T17:06:36 10.0.1.100:36436 52.16.96.58:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.253:60322 162.159.140.167:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.253:59754 162.159.140.167:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:38264 54.73.190.247:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:35326 8.8.8.8:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:36 10.0.1.100:35326 8.8.8.8:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:35 10.0.1.248:55142 8.8.4.4:53 udp Default deny / state violation rule
LAN 2024-09-05T17:06:33 10.0.1.253:56942 172.66.0.165:443 tcp Default deny / state violation rule
LAN 2024-09-05T17:06:32 10.0.1.253:56934 172.66.0.165:443 tcp Default deny / state violation rule
Before I add the second WAN link, I'd like to get this working correctly.

Any suggestions as to how to get the firewall to allow traffic in and out of the 10.x.x.x network?

Oh and I have a sneaking suspicion I know what was happening to cause these problems. I think because of how I had the routes setup before the firewall was sending it's LAN traffic to the switch (VLAN Gateway) for some traffic and sending to the WAN as a gateway for other traffic and that was confusing the hell out of it so it would just drop the packets. Not sure how I did that loop but looking at the logs before I wiped it, i'm almost certain that is what happened. This would totally explain the flapping of all the services like Netflix and Smartthings.

5

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 10:36:01 pm »

That internal gateway - is that a managed switch? Once you got the basics settled, you might want to investigate VLANs and how to use OPNsense as the internal router instead. But first things first.

It is, fully managed L2 switch. A Juniper ex4200. I have my ESXi host piped into it with a bunch of trunk ports, an entire other bunch of physical things like 3d printers and such as well.

6

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 07:09:32 pm »

Things I need by default:
Internal 10.0.0.0/16 networks

Well, no, you do need any such things at home at all. 65K devices is just pure BS. Back to the drawing board.

You are correct, need and want are different I will agree with you on that. However, want ...  I have IP cameras and my kids have minecraft servers, I want them on different subnets to better control who has access to my IP cameras. Do I need a /16, no but ... they are free and it makes my life a little easier managing this little zoo of a network I have.

7

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 04:37:51 pm »

I have 2 Wan links and a couple subnets behind a routed switch.

I overlooked that one. Sorry, I apologize.

Absolutely no worries at all. I sincerely appreciate you jumping in and trying to help!! I've got a post on Reddit up for 3 days with 1100 views and not one reply haha, so I appreciate your assistance. I'll report back once I've zapped the config and reinstalled.

Just to restate, I'm only going to use 1 WAN link to start with and test with that first before doing anything else, then move on to port forwarding, then to internal routing, then to 2 WAN links if all goes well. Is that order appropriate for testing?

8

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 04:27:03 pm »

Nowhere in your initial post did you mention dual WAN or internal routes.

These are advanced topics, most of all dual WAN, and you should know how ip based routing and firewall based policy routing work before attempting such a setup.

If you want the devices in the 10.0.0.0/16 network to be able to access the Internet, you will have to switch your policy under Firewall > Network address translation > Outbound from automatic to at least hybrid and add a NAT rule for that network. OPNsense by default only NATs directly connected networks.

That might explain some of your problems.

I would like to say that I appreciate any and all help, and I appreciate anyone and everyone who takes their time to try and help. 

I did mention dual wan, last sentence first paragraph of the post, also I mentioned it in the "what I tried" section, and it's in the drawing I attached.

I am familiar with how IP based routing works, I can't write ipchains rules off the top of my head but I can have a reasonably intelligent discussion about what a rule would look like based on the requirements. The challenge I am facing now is when I diagram out these flows on a white board, then translate them into a rule in OPNsense I do not get the desired result, and I'm thinking it is because I'm not checking the right boxes elsewhere in the interface other than the "Firewall" menu section.

The services that were spotty lived on a directly connected subnet the 192.168.0.x subnet holds my netflix xbox etc. The route and gateway I setup worked well, all my 10.x.x.x clients could get out to the web and didn't really have too much of a challenge. It was really the stuff on the 192.168.0.x subnet (same as the LAN of OPNsense) that were having issues. That and my port forwarding.

Of note I set the dual WAN to failover, then I unplugged it and deleted the interface, then I reformatted and didn't plug it in or configure it, taking any chance of hairpining happening, or split sessions etc off the table.

I've attached screenshots of the gateway and route I configured.

9

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 03:58:14 pm »

Activate DHCP on the LAN port and set your range, subnet, gateway, and DNS server. 192.168.1.50-192.168.1.200, 255.255.255.0 or /24, 192.168.1.1, 192.168.1.1 in the order listed above for descriptions.

Go to unbound and check the box to allow it to be the system DNS provider, if you don't do this, then you need to set your chosen DNS provider for each client or in the DHCP config (1.1.1.1, 8.8.8.8, etc).

Sorry, no - why? OPNsense will out of the box send 192.168.1.1 as both default gateway and DNS server to the clients via DHCP and Unbound will just resolve any domain you throw at it recursively.

Only mess with settings if you know what you are doing. OPNsense works exactly like a consumer router out of the box.

I will do another reformat install and try again. I don't think I went that far off the reservation but I'll wipe it and start again.

Things I need by default:
Internal 10.0.0.0/16 networks routed to an internal gateway of 192.168.0.254 and I think i figured that out
correctly as traffic was routed from the 10 network out to the internet and back properly. I created a gateway and used it as the target of a static route.

Port forwarding for Plex. Now I've followed the directions to do this, but it just doesn't work reliably (i posted on reddit but no responses yet).

Port forwarding for a deluge server i host for friends and family. This one was tricky as it uses random high ports by default. I've tried even on my Asus to get this working and I can't so I stick it in a "dmz" of sorts on the Asus to allow traffic to flow to and from it on any port. I tried the any, any, to server IP and that didn't work as things were dropped on the way back to the machine at the WAN interface on OPNsense.

I'll give it another go with changing nothing and monitor the logs. Wish me luck!!

10

24.7 Production Series / Re: Home use "easy button" question

« on: August 19, 2024, 03:36:22 pm »

I thought so too, I reverted back to a vanilla fresh install and it still had problems. There are a zillion block entries in the logs that correspond to services that were initiated from inside the network, only to have their response blocked. I suspected it was my dual WAN setup so I removed on of the links before the reformat reinstall, and then left it off after the reinstall.

I too found this odd, but I did see in the logs lots and lots of deny with IP of my tv on various ports.

11

24.7 Production Series / [Resolved] Home use "easy button" question

« on: August 19, 2024, 02:50:26 pm »

New OPNsense user. I know my way around a keyboard but network firewalls I only know enough to do basic things. I have a new opnsense install. Worked great testing it, surfing the web etc, flipped the whole house over to it and all hell broke loose. The default deny policy played havoc with my smarthings, netflix, hubitat, heck even gmail would fail to load at times. I dug through the logs and saw it was even blocking things on the not only the WAN but the LAN interface as well. It was certainly doing it's job for sure, but how would I allow all these home services like streaming and minecraft to work without a giant list of firewall rules making the deny policy all but moot? I plugged my Asus ET12 back in and poof everything worked (I have SOOOOO many questions about the firewall in the Asus router now btw =D). I have 2 Wan links and a couple subnets behind a routed switch. 

Some things I checked/tried:
1) Unbound. In reading through the forums I've seen many threads where unbound is the culprit, so i ran with it enabled and disabled.
2) NAT Reflection, same ran enabled and disabled
3) Port forwarding, I have a plex server tried a standard port forward and it would work, then not so turned all that off
4) IPS/IDS i ran with it on and off. same results.
5) reformatting reinstall put NO plugins or rules or anything in, one WAN link plugged in, and it behaved the same.

I still have all the logs and it is still connected to one of the WAN links, happily sending packets to the bit bucket =D.

A simple example, I created a port forward rule to a single server on port 58112 tcp. The firewall is blocking it, "default deny rule" even though I created a rule to allow it on the LAN & WAN interfaces and everywhere else. 

Picture of my network is attached, couldn't paste it inline.