HaProxy and Crowdsec

[Zenturio]

I'm running OPNsense 24.7 with HAProxy and Crowdsec. I have several services running behind HAProxy some of them with Crowdsec log parsers installed, reporting to the OPNsense Crowdsec LAPI. The firewall bouncer works great with this setup, but I also want to block Traffic at Layer 7 directly on HAProxy. This is where the Crowdsec HAProxy Bouncer comes into play, but I can't get it running on OPNsense. Is there any chance to get the bouncer working?

https://github.com/crowdsecurity/cs-haproxy-bouncer
https://www.crowdsec.net/blog/the-haproxy-bouncer-is-out
https://docs.crowdsec.net/u/bouncers/haproxy/

[JLDC]

Second this for the nginx bouncer as well.  I'm currently considering moving nginx off of my opnsense box just to have this functionality but would ideally like to keep it where it's at.  Handosense was trying to get the HaProxy bouncer working too - https://forum.opnsense.org/index.php?topic=39377.0

[Monviech (Cedrik)]

Its probably easier to get the caddy crowdsec bouncer running with the os-caddy plugin, since it can be compiled into the caddy binary and then you can just configure it in the caddyfile.

There are no outside dependencies.

Adding that would be very straight forward, so if somebody offers a PR I would review it.

https://caddyserver.com/docs/modules/crowdsec

[Patrick M. Hausen]

What use is an HAproxy/Caddy/NginX bouncer if the firewall bouncer kicked in already? Attackers will never be able to contact your L 7 services to get bounced.

[Monviech (Cedrik)]

Well you are right that it does not seem to make sense.

I just feed the logs to OPNsense and it blocks via pf.

[JLDC]

For me, I use Cloudflare proxy and my WAN only accepts inbound from the Cloudflare IP ranges (https://www.cloudflare.com/ips/.  That's all the firewall sees so it can't block by the remote IP that nginx sees.  I could set up the bouncer on each one of my services but would prefer to have it right on the reverse proxy.

[cookiemonster]

I have been wanting also the that  for haproxy New/unknown IP are checked against crowdsec API and have been disappointed that the bouncer is not available for OPN/freeBSD.
I could very well be mistaken but my reasoning is that haproxy has ports exposed via firewall allow rules, so the firewall has handed over the connection to haproxy. Here is where crowdsec would come to do its thing.
Is that incorrect reasoning ?

[Monviech (Cedrik)]

For me, I use Cloudflare proxy and my WAN only accepts inbound from the Cloudflare IP ranges (https://www.cloudflare.com/ips/.  That's all the firewall sees so it can't block by the remote IP that nginx sees.  I could set up the bouncer on each one of my services but would prefer to have it right on the reverse proxy.

Now the usecase is taking shape. The opnsense wont see the proxy protocol or x-forwarded-for header and the block on opnsense firewall level would be useless.

Like written above, trivial to add crowdsec to the existing caddy plugin.

[cookiemonster]

@Monviech would you be OK to share your thoughts on my question please? I would appreciate your input.

[Monviech (Cedrik)]

If you dont mask the IPs of the incoming requests with a CDN you can just use the crowdsec opnsense bouncer, let it parse the ha proxy http access logs and block on firewall level.

I do the same parsing the caddy logs. It works. The CDN usecase is special.

[cookiemonster]

No CDN in front of haproxy so it sounds like it is doable. I'll see if I can figure out how to implement what you suggest. Thanks.

[Monviech (Cedrik)]

Here is how it currently works in caddy with the http access logs for reference.

Its really simple:

https://docs.opnsense.org/manual/how-tos/caddy.html#crowdsec-integration

(thanks @Patrick here who suggested this approach at the beginning

[cookiemonster]

Here is how it currently works in caddy with the http access logs for reference.

Its really simple:

https://docs.opnsense.org/manual/how-tos/caddy.html#crowdsec-integration

(thanks @Patrick here who suggested this approach at the beginning

Many thanks. I didn't even know the docs has had a facelift.

[cookiemonster]

An update on this one. I did follow that suggestion, modified the crowdsec opnsense bouncer but perhaps because the logging in haproxy is so different, it seemed that it had no way to trigger an action.
So I went to see if I could figure out what the haproxy bouncer gets installed and does its thing.
I have been able to install it and configure it and so far both it shows running and I have finished clearing out the last setup failure on haproxy.

An internal test seems to suggest that the action was correctly triggered (in my case I setup a captcha, in reality I signed up to cloudflare turnstile just for this).

I am however not been able to confirm it works for external entities trying to abuse so as to trigger it. Once I've been able to do that and I am more sure that is doing what it is supposed to, I shall be posting a description of what I did.

[cookiemonster]

Update. It seems to be working correctly and serving both bans and captchas.
One last thing to diagnose is a loop I get on the captcha or more specifically the bot-catching "captcha" aka turnstile. Almost there I think.
I've asked crowdsec in various places now just awaiting input.