Messages

After some more testing; the problem is in /usr/local/etc/inc/system.inc:

Code Select Expand

..
            $cmd[] = exec_safe('%s', $rtent['network']);
            if (!empty($rtent['disabled'])) {
                mwexec('/sbin/route delete ' . join(' ', $cmd), true);
                continue;
            }
..
The code only checks if there is a route to e.g. 10.0.0.0/8 which is disabled and if so, it deletes it from the routing table.
It does not consider if the disabled route is to another gateway than the route which is in place. Because we had another circuit in the past there were routes to

Code Select Expand

10.0.0.0/8         
172.16.0.0/12     
192.168.0.0/16     
over another gateway which are disabled in the configuration.
After deleting the obsolete (disabled) routes, everything is working with the new OpenVPN instances, too.
These disabled routes were not in place on my testing device (with the old OpenVPN method), so therefore it was working.
Nevertheless, one could argue about whether this is a bug or a misconfiguration ;) In any case, it now works for us without any restrictions and that's enough for now. Maybe this will help someone with similar problems.

We did a few more tests. With a classic OpenVPN all routes (including the ovpnc routes) stay in place after /usr/local/etc/rc.routing_configure:

Code Select Expand

root@OPNsense:~ # netstat -rn -4|wc -l
      25     
root@OPNsense:~ # /usr/local/etc/rc.routing_configure
Setting up routes...done.
Setting up gateway monitor...done.
Configuring firewall.......done.
root@OPNsense:~ # netstat -rn -4 | wc -l
      25     
root@OPNsense:~ #

Using OpenVPN Instance method;

Code Select Expand

root@OPNsense:~ # netstat -4 -rn | wc -l
      25     
root@OPNsense:~ # /usr/local/etc/rc.routing_configure
Setting up routes...done.
Setting up gateway monitor...done.
Configuring firewall.......done.
root@OPNsense:~ # netstat -4 -rn | wc -l
      22     
root@OPNsense:~ # pluginctl -s openvpn restart
Service `openvpn[95c5c62f-4134-48d1-be28-03720aa2ed02]' has been restarted.
Service `openvpn[cb360e70-8d4e-46b6-ad6f-57582d92a44f]' has been restarted.
root@OPNsense:~ # netstat -4 -rn | wc -l
      25     
root@OPNsense:~ #
As you can see, with the new instance method there are 3 routes which are lost after calling /usr/local/etc/rc.routing_configure:

Code Select Expand

# netstat -4 -rn | grep ovpnc
10.0.0.0/8         172.17.166.25      UGS          ovpnc3
172.16.0.0/12      172.17.166.25      UGS          ovpnc3
172.17.166.25      link#17            UH           ovpnc3
192.168.0.0/16     172.17.166.25      UGS          ovpnc3

I think, this should not happen and is a bug!?

[Is it a bug that the ovpnc-routes are cleared from the routing when editing the table?]

Hi,

I have a stable working S2S OpenVPN tunnel with the new instance method.
The problem is when I try to add / remove routes on the client side the main routing table is reloaded and the routes for the openvpn client are lost!

After starting the openvpn client side the (shortened) routing table looks like:

Code Select Expand

Internet:
Destination        Gateway            Flags         Netif Expire
10.0.0.0/8         172.17.166.37      UGS          ovpnc1
127.0.0.1          link#4             UH              lo0
172.16.0.0/12      172.17.166.37      UGS          ovpnc1
172.17.4.0/24      link#1             U              igb0
172.17.4.1         link#4             UHS             lo0
172.17.166.37      link#11            UH           ovpnc1
172.17.166.38      link#4             UHS             lo0

If I now add a local route, e.g. 192.168.0.0/24 via 172.17.4.100. The routing table changes to:

Code Select Expand

nternet:
Destination        Gateway            Flags         Netif Expire
127.0.0.1          link#4             UH              lo0
172.17.4.0/24      link#1             U              igb0
172.17.4.1         link#4             UHS             lo0
172.17.166.37      link#11            UH           ovpnc1
172.17.166.38      link#4             UHS             lo0
192.168.0.0/24     172.17.4.100       UGS            igb0

The OpenVPN process is still running and even the keepalives are working, so there is no restart of the VPN connection.
But the main routing table is now missing the route for 10.0.0.0/8 and 172.16.0.0/12 (which is configured via OpenVPN server config (CSO)).

If I add these routes manually with

Code Select Expand

route add 10.0.0.0/8 172.17.166.37
route add 172.26.0.0/12 172.17.166.37everything is working again (an OpenVPN restart obviously works,too).

Is it a bug that the ovpnc-routes are cleared from the routing when editing the table?

Hi there,

is there a way to enter/leave the carp maintenance mode via command line or API? I would like to automate some things and did not find anything in the docs or the forum.

best regards,
Torsten

Ah... I was wrong... there is no intermediate ca from LetsEncrypt. But the opnsense  chooses the (wrong)

Common Name: Let's Encrypt Authority X3

as CA for the webgui but the CA is now

Common Name: R3

which is a new CA (Valid From: October 7, 2020)

If I put the right CA inside /var/etc/ca.pem it works, too.... I not sure why and how the opnsense chooses the CA which is put in /var/etc/ca.pem.

So, after deleting the old Lets Encrypt CA from the trust store and reissue a new certificate (force via lets encrypt plugin),  everything works automatically. For me it is ok now, perhaps this helps somebody out there. Thanks for your support!

The chain is properly appended, but only if the parent CA(s) are known to System: Trust: Authorities.

Hmmm...Let's Encrypt Authority X3 (Let's Encrypt) and R3 (Let's Encrypt) were/are in the trusted Authorities. But the full chain is not copied to the *pem file by the opnsense framework (only key+cert).

Perhaps we are looking at different problems with the same effect!?

If I take a self-signed cert for the webgui => no problem with firefox/chrome/edge (on Windows).
If I take the Lets Encrypt cert => ERR_SSL_PROTOCOL_ERROR in chrome+edge, SSL_ERROR_INTERNAL_ERROR_ALERT in firefox
If I add the intermediate directly in the *pem => No problems

tested TLS1.3, no problem with the fullchain

Code Select Expand

...
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=XXXXXXXXX
*  start date: Dec  7 22:01:20 2020 GMT
*  expire date: Mar  7 22:01:20 2021 GMT
...

I think the problem is that when you use Lets Encrypt Certificates via acme.sh the certificate in

Code Select Expand

ssl.pemfile = "/var/etc/cert.pem" (/var/etc/lighty-webConfigurator.conf)
is only the private key + certificate.
The intermediate certifcate is missing. If you put the intermediate certificate into

Code Select Expand

/var/etc/cert.pem and restart the lighthttp it is working for me.
e.g.

Code Select Expand

cd /var/etc/acme-client/home/<MYNAME>
cat fullchain.cer <MYNAME>.key > /var/etc/cert.pem
Restart lighthttpd

Perhaps it is this simple?