Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - CMogen

#1
General Discussion / Re: Dual gateway | Reply-to?
March 13, 2025, 12:26:33 AM
Quote from: dseven on March 12, 2025, 09:44:17 AMIf you took the old router out and only OPNsense remained, it would have **only one** LAN ("int") address, and that would be the default gateway for (all of) your LAN servers. Presumably you would use 10.10.10.1 for that and forget 10.10.10.2.
Fair enough.

For the NAT hack, if you could clarify a few things I think I can run with it.
(I do think the static route on LAN servers would work too, but maybe only with TCP, so I'll start with NAT)

1. Should I manually bind an interface to the vpn device? Docs say this may be required for reply-to and other NAT functions to work properly.
a. Set IP on if, or use whatever passthrough settings from ovpn?
b. Dynamic gateway checked?

2. Outgoing NAT masq- To confirm, this should be the LAN IP, not VPN tunnel IP? (I realize tunnel would require a static route in endpoint OS to work, but LAN I question if ovpn will automatically see/pick that up as return traffic)
a. If LAN: Outbound NAT policy on 1OpenVPN default created if, or on 2manually created interface (if created in step1)?
b. ^ policy: source= Tunnel network > dest= LAN net > translation target= LAN address? [or manually specify 10.10.10.1]

3. Troubleshooting- if the NAT masq works, packets come back to LAN IP, but ovpn doesn't seem to be picking them up.. where/how could I see the logs for discarded packet, no route, etc to find the breakdown?

4. Any general options that should/shouldn't be enabled. I am removing all auto-added pass/deny policies and switching to manual outbound NAT only, but may still have some non-default setting leftover from testing.

I truly appreciate any help, if I can get this working I think I'll be sold and join team OPNsense. Then I'll be happy to support the cause as well; I'm actually not another random FOSS user that expects on-site support and a guided tour, but it is all (frustratingly) new to me and the documentation can be vague at times.


#2
General Discussion / Re: Dual gateway | Reply-to?
March 12, 2025, 12:22:56 AM
Quote from: viragomann on March 09, 2025, 09:17:32 PMIf only need to allow access from the VPN client to your LAN and don't care about the clients source IP, you can simply masquerade the traffic with the LAN IP of OPNsense.
Thanks for that, I'm pretty sure I tried it at some point, but I'm not sure if maybe another (floating?) rule was getting to the traffic first. I plan on deleting all the auto-generated rules and working up and I'll try that again.

Just for the sake of beating horses, and understanding why I'm fundamentally wrong on this, but I still see it as a multi-WAN scenario. Let's say the 7.7.7s are actually two different ISPs, and take the old firewall out, it would look like this all on a single OPNsense:

    ┌────────────────────┐    ┌────────────────────┐
ext │ if1 7.7.7.25 /26   │    │if2  3.3.3.33 /26   │
    │                    │    │                    │
tun │ vpn1 10.9.9.1 /24  │    │vpn2 10.8.8.1 /24   │ two different ovpn instances
    │                    │    │                    │
int │ if3 10.10.10.1 /24 │    │if4  10.10.10.2 /24 │
    └────────────────────┘    └────────────────────┘ 
LANserverA = defgw 10.10.10.1
LANserverB = defgw 10.10.10.2

Without some magical intervention (conntrack, auto proxyarp, noclue) by being on the same device, would I not have the same problem connecting to serverA via vpn2/3.3.3.3 or serverB via vpn1/7.7.7.25?
#3
General Discussion / Re: Dual gateway | Reply-to?
March 09, 2025, 08:06:48 PM
Well, it is a multi-WAN setup (from the perspective of a LAN host), they're just not on the same device, and I think that's most of the problem. If both LAN gateways were on the OPNsense, I'm sure conntrack (or whatever BSD/OPN uses) would auto-magically fix it. The UDP scenario could apply to my situation, and seems like that's exactly what's happening. It's either getting the external IP as a reply to, or it's a weird stateless multi-home UDP thing like this

You're right though, I guess I need to break out Wireshark and try to figure out what's going on. I was hoping someone might point me to some additional docs/workshop/utils, to better explain the flow and when/how those various NAT/reflection/reply options are set. I was trying to avoid manually figuring that out via trial and error and a packet sniffer.

Especially when I've had configs get "stuck" on a few occasions. Like deleting a gateway or interface, and seeing that if's rules still being applied to traffic somehow when I'm testing option F 20 minutes later. Without rebooting between every change, I'm now questioning if what I see in the GUI matches the running config. It's tedious already, and I haven't alt-tabbed to a packet sniffer between every change yet.

#4
General Discussion / Re: Dual gateway | Reply-to?
March 09, 2025, 12:03:08 AM
Quote from: dseven on March 08, 2025, 10:16:01 AMWhat do you mean by "OS route entry"? The static route route need to be added *on the LAN host*.
It was, sorry if it wasn't clear. I said OS meaning Win/Linux route table vs something set in OPNsense.

Quote from: dseven on March 08, 2025, 10:16:01 AMWhat makes you think that the problem has anything to do with VPN encapsulation? If that was the case, why would it work when the default gateway of the LAN host points to the firewall handling the VPN?

Documentation I guess. I assume the same would apply in OPNsense. From https://docs.netgate.com/pfsense/en/latest/vpn/openvpn/multi-wan.html

QuoteThe protocol choice for UDP on IPv4 and IPv6 on all interfaces (multihome) will work properly on all WANs and respond back using the address clients expect.

These other UDP modes in OpenVPN are limited by the connectionless nature of UDP. In these cases, the OpenVPN instance replies back to the client, but the Operating System selects the route and source address based on what the routing table believes is the best path to reach the peer. For non-default WANs, that will not be the correct path or the address the peer used when contacting this VPN.
#5
General Discussion / Re: Dual gateway | Reply-to?
March 07, 2025, 10:43:49 PM
Quote from: dseven on March 07, 2025, 09:49:34 AMI'm going to assume you have VPN clients connecting to these firewalls and they're trying to access some services running on LAN hosts?
Correct.

I believe most of the problem is coming from UDP used for VPN encapsulation. No matter what options I set on the New/OPN firewall, the return traffic always seems to follow the OS's default gateway, even skipping the
Quote from: dseven on March 07, 2025, 09:49:34 AMstatic route for 10.8.8.0/24 pointing to 10.10.10.2 should solve that
OS route entry.

I ended up restoring a backup from a couple days ago because I was so deep in the weeds with adding gateways, reply-tos, NAT, I wasn't sure what all was changed. After restoring the backup and creating a new TCP-based tunnel, it's still not working out of the box, but I assume I need to (re)set a gateway and reply-to for the VPN traffic?

Would this be a situation where I should have a LAN gateway (marked upstream)? Most of the docs advise against that unless absolutely necessary, but I'm not sure how else to bind the VPN to an interface and/or specify the reply-to as the LAN interface IP.
#6
General Discussion / Dual gateway | Reply-to?
March 06, 2025, 11:58:33 PM
Hi, I am in the process of migrating things from a Firebox to OPN. Both firewalls host SSLVPN, and share a subnet with the LAN.

      OLD                         New
    ┌────────────────────┐     ┌────────────────────┐
    │ext   7.7.7.25 /26  │     │     7.7.7.33 /26   │
    │                    │     │                    │
    │tun  10.9.9.1 /24   │     │    10.8.8.1 /24    │
    │                    │     │                    │
    │int  10.10.10.1 /24 │     │    10.10.10.2 /24  │
    └────────────────────┘     └────────────────────┘ 

Things work fine if the target LAN machine's gateway matches the VPN entry point, otherwise the return traffic gets sent to the other firewall. The eventual plan is to change all the LAN to the new gateway, but I was hoping to figure out how to get this to work in parallel, if only to better understand the *sense inner workings and terminology.

I have tried disabling the reply-to WAN option, created an interface for OpenVPN, tried some different NAT options there, even created a static route on a LAN machine for the New tunnel network > New gateway, but I don't see that traffic coming back, so now I'm lost and not entirely sure what OPNsense is doing behind the scenes.

Is reply-to the right setting for this, or one of the NAT reflection WAN general rules?
What IP should the LAN nodes see coming from OpenVPN- the tunnel network IP, default WAN gateway, some other NAT? Does this change with NAT option x?
Do I need a outbound NAT rewrite to make the VPN traffic source look like New gateway perhaps?

If someone could break down the NAT automatic outbound/1:1/reflection/reply-to terminology and use cases I would really appreciate it. I've re-read the documentation on each several times and still can't get my head around what they actually do or when/where that would be applied, especially with ovpn in the mix. TIA

#7
Thanks for all the helpful replies and diagnostics. In the end I believe it was an installation issue. Apparently I downloaded the OPNsense installer instead of pfSense. Once the correct installer was used, ACME cert issuance breezed right through. Like a tumbleweed blowing through a ghost town. /allegory
#8
Well, this has been an absolute nightmare. When performing the steps above, it seems to at least sync the cert with the issuer, but afterwards all cert options are greyed out except "Import cert (signed by CA)" like it's still waiting. No option to reissue, and I don't think the cert update is being pushed anywhere after the save.

I found the key/cert in /var/etc/openvpn/instance-xx.conf was still using the original (pre-create CSR/import above) pair. Restarting OpenVPN doesn't do anything, you have to change the cert to something else, then back to the same cert the instance was using to regenerate the proper config file.

Still getting a (different) TLS handshake error.. could this have something to do with when the original certs were generated, possibly on an older version https://forum.opnsense.org/index.php?topic=42220.msg209001#msg209001 ?

If so I'm just going to wipe this device and start over. If this is par for the course though, or the current state of PKI/trust/509/whatever, please let me know and maybe I'll put PF on it instead.

#9
Apparently I'm having the exact issue described here:  https://forum.opnsense.org/index.php?topic=41943.msg206958#msg206958
Anchored post steps fixes the "cannot verify CA" part.

In trusts > certs, the ACME cert shows up as self-signed.
Edit cert > save (no changes) says "CA key not present"
Switch to create CSR > Import works. Cert now shows with LE CA. *however* Under Trust > Authorities, the LE CA shows up as self-signed, and no key present in edit properties. (pic)

Tested with Linux client verb4, dies with:
2025-02-24 15:39:51 us=578829 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2025-02-24 15:39:51 us=583757 OpenSSL: error:05800074:x509 certificate routines::key values mismatch:
2025-02-24 15:39:51 us=583815 Cannot load private key file [[INLINE]]
2025-02-24 15:39:51 us=583934 Error: private key password verification failed
2025-02-24 15:39:51 us=583997 Exiting due to fatal error
#10

OPNsense 25.1.1-amd64
FreeBSD 14.2-RELEASE-p1
OpenSSL 3.0.16

Hello all,

I've been fighting with this for a couple days now and I'm starting to question my own sanity. Every time I use a LE cert via ACME, I get:

VERIFY ERROR: depth=1, error=unable to get issuer certificate: C=US, O=Let's Encrypt, CN=R11, serial=184083759606652600789093070426744763640
Feb 23 10:10:31 AM: OpenSSL: error:0A000086:SSL routines::certificate verify failed:
Feb 23 10:10:31 AM: TLS_ERROR: BIO read tls_read_plaintext error

When connecting with OpenVPN connect and also Viscosity. Two different Win10 machines.

This was happening in the last 24.x.x and also in 25.1.1. I've deleted/reissued the cert several times, nuked the OpenVPN instance and recreated it, un/reinstalled ACME client, along with everything mentioned in relevant forum posts:
https://forum.opnsense.org/index.php?topic=24973.0
https://forum.opnsense.org/index.php?topic=12060.0
https://forum.opnsense.org/index.php?topic=24950.0

Delete, add/import CA manually, change update repo/reinstall ACME, inline cert info in .ovpn, practically every VPN cert-related option (verify remote, verify client, depth, etc) I've honestly lost track at this point. I'm not seeing anything in (debug) logs pointing to any issues, everything seems to be happy, but clients are not getting the proper CA chain. I'm to the wipe/reinstall OPNsense on a Sunday afternoon desperation point, so I would appreciate any other options!