OPNsense Forum

English Forums => Web Proxy Filtering and Caching => Topic started by: lilsense on July 18, 2021, 02:21:09 pm

Title: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 18, 2021, 02:21:09 pm
Would someone be able to convert this video into Opnsense HAPRoxy. I am having a hard time making mine to work.

https://www.youtube.com/watch?v=LlbTSfc4biw
Title: Re: OPNSense HAProxy and Cloudflare
Post by: sorano on July 18, 2021, 10:07:35 pm
Well, it seems a bit much asking someone else to create a video for you but I'm proxying a domain from Cloudflare to HAProxy and the Cloudflare settings are pretty much the same as in the video.

I have not bothered to do the Full (strict) SSL/TLS mode but the Full mode works fine for me.

If you already have a proper HAProxy setup it should not require any additional configuration in HAProxy except maybe creating an ACL that allows Cloudflare IP's only.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: fabian on July 18, 2021, 10:10:56 pm
As the original developer of the nginx plugin, I know that there is also a specific header as the real IP source.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 19, 2021, 11:33:07 am
I am not asking for a new vid. I am asking for help setting up the HAP to work. I followed the vid and I am unable to start the HAP.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 19, 2021, 12:21:58 pm
As the original developer of the nginx plugin, I know that there is also a specific header as the real IP source.

do you have a process to go this with nginx as opposed to HAP? I think nginx uses more resources, but I think it will be ok.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: sorano on July 19, 2021, 02:47:05 pm
Take a look at this guide to get HAProxy up and running:

https://forum.opnsense.org/index.php?topic=23339.0
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 19, 2021, 04:38:55 pm
one issue is that it uses Let's encrypt instead of the Certs on the Opnsesne in the Trust section.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: sorano on July 19, 2021, 06:36:10 pm
Why is that an issue though?
That's what I'm doing and it works with Cloudflares Full mode.

Otherwise you can generate a CSR under System - Trust - Certificates, put that in Cloudflare to get your cert and then import your cloudflare cert in OPNsense and use that in HAProxy.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: fabian on July 19, 2021, 07:10:34 pm



do you have a process to go this with nginx as opposed to HAP? I think nginx uses more resources, but I think it will be ok.

https://github.com/opnsense/plugins/blob/master/www/nginx/src/opnsense/mvc/app/models/OPNsense/Nginx/Nginx.xml#L702

You can do that in HAProxy as well but it is maybe not provided by the plugin itself.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 20, 2021, 12:39:31 pm
I am not sure what I am looking at that XML.

I found this:
https://www.loadbalancer.org/blog/how-to-add-cloudflare-in-front-of-a-load-balancer/

would this be done on the Opnsense?
Title: Re: OPNSense HAProxy and Cloudflare
Post by: fabian on July 20, 2021, 05:06:05 pm
This is a header sent by CF to tell your app who is talking to it on the other end.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 20, 2021, 09:41:01 pm
can this be done with wireguard?
Title: Re: OPNSense HAProxy and Cloudflare
Post by: fabian on July 20, 2021, 10:50:38 pm
can this be done with wireguard?

no that is a VPN. Cloudflare does not work if you use a VPN to bypass it.

If you use Cloudflare with your HTTPS, you will have the following connection:
Client Out -> In Cloudflare (Blackbox) Out -> In OPNsense Load Balancer
Since your LB will get the IP of Cloudflare as remote IP, your logs will be quite useless unless you configure a real IP source to log the right IP address.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 22, 2021, 01:26:51 am
Take a look at this guide to get HAProxy up and running:

https://forum.opnsense.org/index.php?topic=23339.0

I am trying to follow this using the lets encrypt, but it seems that cloudflare DNS is not working and I am unable to get the ssl working.

I get the following error: AcmeClient: domain validation failed (dns01)
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 22, 2021, 02:07:14 pm
One thing is so scattered was the DNS resolve for cloudflare... it looks like the Let's Encrypt is trying to use/create TXT witha certain value. How do I make this work in cloudflare. when I update this information the data changes.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: sorano on July 22, 2021, 04:22:12 pm
You must create an API token that has DNS permissions in Cloudflare and then configure that token for your validation in OPNsense.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 22, 2021, 04:56:58 pm
so I ran:
./acme.sh --issue --home . -d 'domain.com' --dns dns_cf --debug 2

and got this:

[Thu Jul 22 10:49:09 EDT 2021] Can not find dns api hook for: dns_cf
[Thu Jul 22 10:49:09 EDT 2021] You need to add the txt record manually.
[Thu Jul 22 10:49:09 EDT 2021] Add the following TXT record:
[Thu Jul 22 10:49:09 EDT 2021] Domain: '_acme-challenge.domain.com'
[Thu Jul 22 10:49:09 EDT 2021] TXT value: '5PDYWLn6JD8_some_value_M4clBfO8vkwkgg'
[Thu Jul 22 10:49:09 EDT 2021] Please be aware that you prepend _acme-challenge. before your domain
[Thu Jul 22 10:49:09 EDT 2021] so the resulting subdomain will be: _acme-challenge.domain.com
[Thu Jul 22 10:49:09 EDT 2021] Dns record not added yet, so, save to ./domain.com/domain.com.conf and exit.
[Thu Jul 22 10:49:09 EDT 2021] Please add the TXT records to the domains, and re-run with --renew.
[Thu Jul 22 10:49:09 EDT 2021] _on_issue_err
[Thu Jul 22 10:49:09 EDT 2021] Please add '--debug' or '--log' to check more details.
[Thu Jul 22 10:49:09 EDT 2021] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
[Thu Jul 22 10:49:09 EDT 2021] _chk_vlist
[Thu Jul 22 10:49:09 EDT 2021] Diagnosis versions:

what do I need to add to the conf file?  It looks like it has a certain format.

This may be a bug, as I see this in the script attempting to use http-01...

challenges":[{"type":"http-01",
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 22, 2021, 06:10:44 pm
I decided to uninstall the letsencrypt and used the CF origin and CF cert directly. Now back to the original issue of setting up HAP. LOL.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: sorano on July 22, 2021, 06:42:29 pm
Why are you doing stuff from cli?
Cert and validation is all configured in the webui from lets encrypt plugin.

Use the staging environment until all is working then switch over to production.

Looks like you are making life hard for yourself.
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 22, 2021, 06:45:34 pm
Everything is done thru GUI with no success...

So, here's something funny... After uninstalling letencrypt, HAProxy started to working but now it's stopped with this error...

   [d7908357-7f95-4ada-83be-6e8a3c85c3e7] Script action failed with Command '/usr/local/opnsense/scripts/OPNsense/HAProxy/syncCerts.py actions --output bootgrid --page-rows '10' --page '1' --search '' --sort-col '' --sort-dir ''' returned non-zero exit status 1. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 479, in execute stdout=output_stream, stderr=error_stream) File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/OPNsense/HAProxy/syncCerts.py actions --output bootgrid --page-rows '10' --page '1' --search '' --sort-col '' --sort-dir ''' returned non-zero exit status 1.   
2021-07-22T12:42:19   configd.py[11318]   [2f872d65-6a03-4abb-9780-5a40222eee14] Script action failed with Command '/usr/local/opnsense/scripts/OPNsense/HAProxy/socketCommand.py show-servers --output bootstrap --page-rows '10' --page '1' --search '' --sort-col '' --sort-dir ''' returned non-zero exit status 1. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 479, in execute stdout=output_stream, stderr=error_stream) File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/usr/local/opnsense/scripts/OPNsense/HAProxy/socketCommand.py show-servers --output bootstrap --page-rows '10' --page '1' --search '' --sort-col '' --sort-dir ''' returned non-zero exit status 1.

It's like hitting a cinder blocks one at a time... LOL
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 23, 2021, 12:33:36 pm
OK.
So I cleaned up all the HAProxy, uninstalled it and reinstalled it back and went thru the tut: https://forum.opnsense.org/index.php?topic=23339.0

All was fine until the last portion of the step 9. Public Front end.

I am not using the let's encrypt. And now HAProxy will not start...
Title: Re: OPNSense HAProxy and Cloudflare
Post by: lilsense on July 23, 2021, 04:10:19 pm
here's the HAP config:


After the patch update today... all is well... It's up and running. :)