Topics

Apologies if I've missied an existing solution somewhere... I did do searches on this and found a thread back around the 22.1 RC timeframe that 'might' be related but did not seem to offer a conclusive remediation and might not be the same issue I'm experiencing...

Background: I have been running OpnSense as a VPN ("only") gateway for the past couple years on a single NIC Intel NUC so everything other than the interface assigned to LAN is a tagged vLAN... It has multiple WAN links (1gb, 300mb, LTE) and multiple VPN links (one for each WAN) all handled by an L2 vLAN switch... The performance has been excellent with full gigabit throughput from a physical PC on the LAN to internet hosts with consistent speed test results on 21.1->23.1

I am now building out a Hyper-V VM with a slightly different configuration (1 WAN link, 2 VPN tunnels, LAN + several additional vLANs that will be firewalled and have limited or no access between them)... The WAN and LAN are on separate virtual NICs defined on the VM at the HV level and I have a Win 10 VM with a single vNIC on the LAN side on the same vSwitch as well as a physical PC on the LAN side to test from...

Upload speeds are fine but download speed is about 25-30% of what I would expect...

Details:
* Win 10 VM is on the same virtual "10 gb" switch as all the OpnSense vNICs/vLANs
* vSwitch tied to a physical 3 NIC "team" (LAG) between the host server and L2 vLAN switch in the server rack
* Rack switch has 1gb uplink to main L2 vLAN Switch near ISP router
* ISP router has 1gb connection to main L2 vLAN Switch
* both physical switches have plenty of backplane bandwidth and are not handling excessive traffic

Thoughts:
* no bottleneck on 10gb vSwitch
* no bottleneck on 3gb LAG
* data transfer between PC on main switch and other (Win Server 2019) VMs on the same vSwitch/Physical switch are fast
* some "potential" limitations of 1gb fiber link between switches but should not limit downlaod to 25-30% of normal

My gut says there is something about OpnSense or FreeBSD that isn't working well with my Hyper-V setup as I've done many other things with this host and set of switches (even using multiple vLANs and other virtual router configs) -- I have not done a lot of deep granular tweaking of Hyper-V network settings other than turning off VMQ on the physical NICs (they are Broadcom and turning that off has long been recommend on these NICs) and I'm not very familiar with low level settings on OpnSense or the underlying networking of Hardened BSD...

Hoping someone else has already experienced this and has a fix for me or that this does in fact relate to whatever changed (and caused issues) in 22.1 RC and there is a remedy via tweaks on OpnSense or HV (or both)...

Please advise!

Background: I've setup OpnSense with multiple WAN gateways (dual internet + LTE fail-over) with a VPN tunnel (via public VPN provider) on each WAN link... I have a WAN_Gateway group and a VPN_Gateway group setup with the appropriate Tier1/Tier2 gateways and policy based routing via Firewall rules on LAN... all of that is working fine...

I am working on moving away from PiHole on separate device to AdGuard Home on the OpnSense... I have everything working EXCEPT I cannot figure out how to route the DNS queries from AdGuard to public DNS via the VPN_Gateway group (or even via a specific VPN gateway)... For PiHole (separate device on LAN) I just needed a rule with source being PiHole IP... But, for AdGuard (on the firewall itself), I can't get a rule to work (LAN or even floating)...

I can see queries going out in the live view of Firewall logs (via the "let out anything from firewall host itself" rule) and it shows ">WAN {LAN interface IP} {upstream DNS IP}" ...

I've tried rules on LAN, WAN, and floating... I fear I'm missing something silly... Hopefully this is in fact something simple... I don't fully understand the relationship between AdGuard and OpnSense with AGH running on the device itself... but it does everything I want, the way I want, except for routing the upstream queries over the VPN (preferably using a VPN Gateway group that load balances those tunnels)...

FYI: I am using a DNS-over-TLS connection to the upstream DNS servers... but I want to obfuscate both ends... DoT insures the payload of query/answer is not intercepted by ISP or snoopers in the routed path... VPN insures upstream DNS is not aware of the true origin of the query and load balancing across 2 VPN tunnels creates further obfuscation as well as redundancy (fault tolerance)... Again, I had this all working with PiHole (which sent queries through firewall via a DoH proxy on the PiHole using docker with PiHole & Cloudflaird) so the only real difficulty is that the LAN rule I was using for that does not seem to work with AdGuard running on the firewall itself...

I have multiple VPN clients configured to connect to separate servers in different areas (US-Atlanta, CA-Vancouver, UK-London, etc.) and want to route traffic via these specific tunnels (exit points) based on source network or, in some cases, service/ports...

All these clients are using the same VPN provider and while each connection gets assigned a different virtual IP (10.x.x.y) all connections appear to be assigned the same gateway IP (10.x.x.1) no matter which server is being connected to...

The gateway created for the 1st VPN interface will display the IP for the virtual network (10.x.x.1) but the 2nd connection will display no IP for the VPN gateway even though the connection is "up" and all other aspects appear normal and functional...

I'm assuming this behavior may be related to multiple tunnels being on the same "virtual subnet" and all having the same IP for upstream gateway (i.e. VPN1 = 10.1.2.101, VPN2 = 10.1.2.102, Gateway for both = 10.1.2.1)...

I'm not sure if this behavior is normal/expected, if I've found a bug or limitation, or if this setup is just not viable on OpnSense...

Has anyone set something like this up using a public VPN provider?

All connections are OpenVPN using UDP and each connection "works" as long as I only try to use one at a time... Is there any work-around for this scenario? Is there, for example, a way to route via the assigned VPN interface instead of by gateway?

I run a multi-wan setup with a 300 megabit and a 1 gigabit link. I have 2 separate VPN providers with the client configuration bound to their respective WAN interfaces. I use a Gateway Group and LAN firewall rule to direct traffic down the tunnels on each WAN link in a load balanced configuration with the respective gateways configured to use "weight" to balance more load toward the gigabit link, etc. I have not turned on Sticky Connections in Firewall/Settings/Advanced...

This setup has worked well through testing and produced good numbers via speedtest.net and, until now, I've not had problem with websites other than those that actively try to detect and/or block use of VPN providers (or block due to incorrect GeoIP data). In other words, the load balancing and potentially changing IPs under the covers have not presented a problem, in general.

However, I've now run into a single website (retirement fund custodian) that uses OKTA for MFA and frequently either fails the login process or kicks me off the session... After further research I believe this is because the mechanism they have setup is not tolerant of IP source changes during the session, etc.

Using Sticky Connections should resolve the issue for the site in question but it will also prevent bandwidth aggregation and decrease the benefit of load balancing for all other sites since it is a global (all or nothing) setting... So I'm now trying to brainstorm a solution that would allow me to resolve the issue for the site in question without losing the benefits for all the other sites...

I know I can configure firewall rules such that a particular source IP bypasses the load balancing and is sent down only one specific tunnel... In theory, this could be done for a destination IP as well but the trouble is websites like this have multiple IPs and tend to reference other sites with multiple IPs. I know I can create an alias that is populated via DNS lookup as well but at the very least I'd have to determine all FQDNs used by both this site and OKTA to resolve the issue that way...

So can anyone else think of a creative solution where I don't have to enable sticky connection for all traffic but can force traffic using this site to be sticky (or to only traverse one of the two WAN/VPN routes)?

Looking for something as close to "best of both worlds" as possible here...

Thanks!

Hello,

I'm wondering if there is a guide or check list (or set of bullet point recommendations) around securing OpnSense for full production deployment. There are obvious things like not turning on SSH (or making sure it is setup to be super secure and only accessible via the LAN), setting a strong password on root, possibly setting up MFA, etc. but I'm wondering if there are more considerations and/or if anyone has or knows of content that addresses this question specifically.

Also curious if most of you access the OpnSense using 'root' or if you tend to lock that down and access the firewall using a separate admin login once in production?

Hey everyone.

I'm new to the forum and new to OpnSense (but not new to firewalls, networking, etc.) and this is my first post here. I have done some searching both via google and on the forums here and was surprised not to find much on this topic. I hope this is the correct place to ask this question and that I have not missed something obvious either in the configuration or in my searches...

I've been working with my nephew to deploy OpnSense for a "home lab" scenario. This started as an exploration of open source firewall alternatives and now I'm ready to put something into "production" with real traffic and devices behind it and, as such, I'm starting to lock things down and harden the configuration...

As part of this I wanted to configure an admin user (for my nephew) with read only privileges that can view all of the pages and logs and such but cannot make changes to the configuration. I did this for him on my SonicWALL originally so he could learn stuff and even help me troubleshoot (but without me having to worry about him making unauthorized changes or playing whack-a-mole trying to solve a problem)...

So far, I've created a group called "view" and started setting up GUI privileges but it seems like certain pages allow editing or seem to be all-or-nothing (see and edit or don't see at all)... Perhaps I don't understand exactly how this works or what the limitations are but before I spent too much more time on it I thought I'd ask if there is a guide or set of recommendations for creating a read only "admin" that can see everything but not change it.

Perhaps there is a simple way of doing that I'm just missing?

Also, on a somewhat related note, I'm looking for a guide on how best to harden the OpnSense configuration. I've set a strong password on root, created a separate user with full admin privileges for myself, made sure SSH is not enabled, etc. but I'm still fairly new to OpnSense and not feeling 100% confident I have not missed something. I've also not attempted to setup MFA at this point (yet).

Thanks.