Messages

I found a hint. When I wake the OPNsense box using WOL it will thereafter_not_ shut down but reboot when executing a

Code Select Expand

shutdown -p now. After that reboot, executing a

Code Select Expand

shutdown -p now has the expected result: the box shuts down ;)

This is not likely an OPNsense specific issue. But if anyone has a solution, I am all ears!

Hi all,

Any idea what might be causing this? In about 50% of the tries, power off actually reboots the Qotom box.
Any logs I can examine?

I like wireguard too. A lot:-) but it's still not widely supported but I agree, for speed this is the one to watch.

I had to delete the default route because each time you setup a Openconnect VPN, it adds itself as the new default GW. This does not affect PBR as you remark, but I don't want the 'regular' LAN-hosts to go through either a VPN. That is why I delete the default route openconnect adds and just add the one to my ISP again.

A 2nd reason is because of the renaming of the openconnect interface The current VPNC script does not restore everything back to normal. Normal being no leftovers in the routing table after it disconnects. The cause of this, is the renaming. The script should look for the new name instead of the old one.

My next step is playing with the VPNC script too. It shouldn't be too hard to stop it from adding a default GW.

Just had the time to write up my overview. See attachment! :-). I believe my manual actions can be scripted easily but I'm not very good at scripts. At all.

[not]

It doesnt set up the default route, it's the server pushing you this route.

Oke I got it. Final hurdle taken at last. If you do not allow openconnect to setup a default route (which I do not want as I run multiple tunnels), you need to define a gateway in Opnsense that you can use in the PBR firewall rules. But that's just pointing out the obvious.

This is the undesired situation:

Code Select Expand

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.65.244.84       UGS      ocvpn0
10.65.0.0/16       10.65.244.84       UGS      ocvpn0
10.65.200.84       link#9             UH       ocvpn0


If we remove the default route, and restore the one to the isp, this would be the new desired situation

Code Select Expand

Internet:
Destination        Gateway            Flags     Netif Expire
default            92.xxx.xx.1       UGS        igb0
10.65.0.0/16       10.65.244.84       UGS      ocvpn0
10.65.00.84       link#9             UH       ocvpn0


However, here is the caveat: if you define the GW in the gui as being 10.65.244.84 it won't work. As 10.65.244.84 points to its own ocpvn0 interface address. That does not compute for Opnsense so it just does not use it. We do know the actual GW  (=the remote vpn server) is somewhere in the /16. So filling in any other IP than 10.65.244.84 that is within the /16 fixes the routing.

So it's just a trick to get packets to the ocpvn0 interface. The real gateway does not matter it just can't be the interface address as I stated above.

After lots of tinkering I can start on the how to now:-)

[isn't]

Yes I'm not questioning that :D. But it is the vpnc-script that sets up the routes in the actual routing table (routes that the server pushes) My point is that the real default gateway ip isn't in the routing table. Something more is going on behind the scenes here.

I did not immediately see this default route in the verbose output in openconnect but I'll look a bit further.

When you send me a list of all ciphers I can add a dropdown list

I just completed testing with multiple instances after getting some alone time with the router:-). I'll send you a write up as pdf with all the steps I took this weekend. There is one thing that I don't quite grasp so I'll spill the beans on that :

Code Select Expand

ocvpn0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1322
options=80000<LINKSTATE>
inet6 fe80::4262:31ff:fe00:c081%ocvpn0 prefixlen 64 scopeid 0xb
inet 10.23.244.84 --> 10.23.244.84  netmask 0xffffffff
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
groups: tun ocvpn
Opened by PID 89427

When you run openconnect with your plugin it sets up the default route (which I obviously don't when using multiple instances).
When trying to figure out what the Def. gateway was, I did a netstat -rn. So the default gw pointed to 10.23.244.84, which is also my interface ip as you can see in the code fragment just above.

Apparently, 10.23.244.84 is not the real gateway. When doing a traceroute the next hop was within the /16 in the routing table which was added by the vpnc-script:

Code Select Expand

traceroute to google.be (172.217.13.163), 64 hops max, 52 byte packets

1  10.65.0.1 (10.65.0.1)  107.129 ms  109.283 ms  107.764 ms

Code Select Expand


Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
10.65/16           10.23.244.84       UGSc            0        0   ocpvn0

So I don't understand why the real gw 10.65.0.1 is not in the routing table. But in the opnsense gui for setting up the far gateway, I had to fill in the real one I found using traceroute. So this is not a parameter for establishing the session, but it is essential to configure the real gateway in the Opnsense gui manually:)

It is best to let DTLS negotiate the cipher itself with PSK-NEGOTIATE. On Linux this behaves. On BSD it does not for some reason. So the only valid strings I found in the RFC are these:

dtls-ciphers=OC-DTLS1_2-AES128-GCM
dtls-ciphers=OC-DTLS1_2-AES256-GCM
dtls-ciphers=PSK-NEGOTIATE (which is the default and results in an MTU-mismactch)

So the option is an override if you either want to use GCM explicitly or in this case when the negotiation botches up
You could go openvpn style and just provide a free text windows where you can put parameters. Just an idea.

[MTU-issue 1320 (client) vs 1340 (server)]

Here is a status update:

MTU-issue 1320 (client) vs 1340 (server): SOLVED
The patch won't be required. I found out that the client is presenting these ciphers:
Cipher Suites (4 suites)
    Cipher Suite: TLS_PSK_WITH_AES_256_CBC_SHA (0x008d)
    Cipher Suite: TLS_PSK_WITH_AES_128_CBC_SHA (0x008c)
    Cipher Suite: TLS_PSK_WITH_3DES_EDE_CBC_SHA (0x008b)
    Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)

And the server is picking the first while the server itself prefers GCM-ciphers. This is the cause of the MTU-mismatch. The GCM-servers have lower overhead so the server can get away with 1340 bytes and the client can't as the CBC ciphers has a higher overhead. Is it a negotiation bug? I'd say yes.  I've put it on the openconnect mailing list.

Anyway the fix is to use the option:

Code Select Expand

--dtls-ciphers=OC-DTLS1_2-AES128-GCM or

Code Select Expand

  --dtls-ciphers=OC-DTLS1_2-AES256-GCM


The VPNC-script: I'm going to manual edit the route table for now. That manual edit will be replaced by altering the script later. This has been done before for various purposed. This isn't really a blocking factor.

Multiple tunnels + routing set up:: the gist of the matter. I'm going to set this up hopefully over the weekend with a full test. If that works it would be super if you could make multiple instances happen (no pressure intended!) ;D

Will do. I need to incorporate a patch first for the MTU-issue and recompile openconnect on BSD so that will take some time as I've never done that before.

I suspected that. Unless you assign them, then they pop up.

Well I'm still looking into what exactly happens. If I set the VPN-GW ip (the private one of course) and I NAT to that...Surfing just works but traffic skips the VPN. So I'm going to give it one more shot and see whether I can pinpoint what happens. But since the VPN private IP is dynamic, it won't be a reliable solution.

Hmm I did try that with interface OpenVPN (via gui) but it didn't work. Perhaps I missed something. It definitely was not NATTING

Yes, I've managed to test it already. I've tested it using CLI, adding the FW rules and NAT (manually) for running a 2nd interface. But I need to be able to use tweaked VPNC-scripts that don't add a def. GW.

Basically passing more parameters like the way you would do when invoking openconnect via CLI.

Once the MTU-issue is resolved (which I'm confident will happen as openconnect has active development) I can remove my temporary Openvpn setup. 

It's allowed, but the one-liner makes the same. You dont have to assign interfaces (which can brake system). Create a gateway and mark as far (since there is no IF) .. then you can set in firewall rules as you described yourself..

I can see how to use the GW in the LAN FW rules. But what is the translation target for outbound NAT? You fill in the virtual address of the tunnel? This does not seem to work for my as I still get to see the public ip from my ISP.

Would you reckon it is much work to include the option to create multiple interfaces (like with Openvpn) and provide the option to pick a custom vpnc-script per interface? That way the routing can be custom tailored for those that can edit the vpnc script?