Messages

OK.
My script worked eventually, after some hours of waiting...

Code Select Expand


var d = $.Deferred(), prom = d.promise(), times = 0;
d.resolve();
$('#rules tr[data-row-id]').each((i, r) => {
   if(times < 0) return;
   prom = prom.then(function() { return $.post('/api/ids/settings/delPolicyRule/' + $(r).attr('data-row-id')).promise(); } );
   times++;
   if(times > 100000) { times = -1; prom = prom.then($.post('/api/ids/service/reconfigure').promise()); return; }
});
prom = prom.then(function() { return $.post('/api/ids/service/reconfigure').promise(); });


You need to :

• Go to this URL : /ui/ids/policy#rules
• Select to show ALL rule adjustments
• Open the dev tools console in the browser, go to the console tab, paste it in and hit Enter
• Monitor the requests in the Network tab to see when it will finish. It can take several hours if you have many rule adjustments.

Eventually, it will delete everything and then you can start all over....
Now, I went from 24000 rules adjustments to only 3!

[UPDATE:]

I desperately want to switch to policies from rules, but the UI tortures me.
I have too many active rules (~24000) and deleting them from the UI is nearly impossible.
I have even written a JavaScript script that deletes them based on UI data, but it works slowly as well (the server is the bottleneck). It will take eons to finish.
Is there any good way to reset the IDS rules and start from scratch?

Also, I fail to understand the difference between the old "Rules" tab and the new "Rules adjustments" tab.
They look similar, but they don't contain the same amount to entries.
So, I am completely lost here. Probably some UX touches are needed there.

UPDATE:
Hm...
Maybe editing the SQLite DB here might do it:
/usr/local/etc/suricata/rules/rules.sqlite
Or it might just destroy my firewall... :\
It seems to contain the rules though... The Administration section rules, not the Policy ones...

OK, I think the update problem was caused by a misconfigured network interface.
Probably, only the error message was wrong.
The web interface seems functional too, so that's good.

Nice to hear that there is nothing wrong once you get it installed.
I will give it a try too!  ;)

Hello.  :)

Just FUI, the installer for version 20.7 seems to still hang at "Select Task" on Hyper-V Server 2019 Generation 2 VMs.
This was reported in the past here, for an older version of OpnSense, and the same behavior still applies:
https://forum.opnsense.org/index.php?topic=10136.0

Also, when the installer finished (after applying the hacks mentioned in the linked post) and tried to reboot, I remember getting an input/output error message. And I had to reboot manually.
Also, when I try to update OpnSense from the management console, I always get this message:

Error updating repositories!
A firmware update is currently in progress.

It is a shame that OpnSense doesn't work in Hyper-V.
I really liked OpnSense and it's abilities.
And Hyper-V is quite amazing too, being 100% free and 100% controlled with PowerShell.

Any hints or ideas are welcome.

Sorry for the obfuscation.
But don't worry, you did help, a lot. :)

I remember trying to reorder the rules with drag and drop but I think the weird effects that I saw (there is an offset between your mouse and the dragged item) made me think it doesn't work.
I also remember removing all the rules and re-adding them in the correct order, but when I reopened the modal they were back to the previous order. I cannot reproduce this today, which is weird.

So, I have managed to make it work eventually!
I reordered the rules to put ACME first and did what you say with the frontends, almost.
I have a frontend for the ACME and the upgrade of HTTP to HTTPS, and another frontend for the site.

It seems to work great! :)
Thanks a lot.

Just took a look at my old pfSense box.
There, I used a primary and some shared frontends, sorted accordingly.
The primary does the SSL offloading and the ACME, and the shared are for each different backend/site.
See attachment.

It seems that none of that is supported on OpnSense (prinamy/shared frontends or sorted frontends).  :(

Here is the config (I did some renames to it):

https://mega.nz/file/JF1gBYwI#cbi9rhNeA0Te1I2e4ph3RYo9RHxKwlqVDOHxCf_Au2g

So for Let's Encrypt you are doing the HTTP-01 challenge from a certain backend?

Yes, the backend generated by the Let's Encrypt plugin.

Alternatively are all of these domains managed by the one account?

Yes, all domains are managed by one account.

Would it potentially not be better to do a DNS-01 challenge and then you don't need to worry about trying to pass the challenge through HAPROXY

I can look into that, but I have never used DNS-01 before, and I don't know if I will have other issues.
Most likely I will have problems with it, since most of my domains are on free dynamic DNS and I am limited on my options.

However, shouldn't the ACL order matter?
I was expecting to be able to sort the HAProxy checks somehow, from the OpnSense GUI.
Notice that in my attached file the ACME ACL is last, while it needs to be first.

[Update:]

Hello.

I want my OpnSense to have a single frontend doing the SSL offloading, and some domain-based rules that forward traffic to the appropriate backend, according the the domain name on the request.

However, I also want Let's encrypt challenges for all domains to pass through HAProxy and get directed to a single backend, so I have added a (path-based) rule for that too, amongst the domain rules.

Normally, I would expect this rule to be the first on the list of rules on the HAProxy config file for things to work, but it seems that it is placed last and the ACME challenge fails trying to hit the wrong web-server (at least, that's what I think makes it fail, since I see the traffic pass though the firewall just fine).

Looking at the UI, I don't see any way to give priority to rules on a frontend or priority to frontends (a.k.a. public services).

Am I missing something or this is a known lacking feature?

I have managed to do this in the past on a pfSense firewall.
I have the impression there was some way to set the order of some things there, but it might be just my impression.
However, listing the rules (marked with ACL in the config) in different order and putting the ACME rule first would have solved the problem, right?
I am asking because I am not an expert on HAProxy either...

Thanks.

Update:
On a second review, I should be probably using the term "condition" or "check" instead of "rule" here.
Hopefully, you understand what I am trying to say.

OK.
I found it:

Using 127.0.0.1:6945 as a listening address made it work.
It seems the hostname I was using was pointing to my public IP and that created a mess.
I remember that working differently on pfSense, but maybe I am wrong.
Or probably there is a setting for this, as mentioned on this Linux-related post: https://stackoverflow.com/a/41009557/964053
Anyhow.

Thanks for nothing. ;)
Your firewall abilities seem better, but your community seems pretty DEAD! :D
It feels I am the only one here.

But thanks for the firewall.
I might decide to contribute some code to it one day...

Any idea how to troubleshoot this?
Does HAProxy's frontends usually work without manual intervention?
Why would I get such an error?
There is nothing listening on that port already, so it should work. :(

[UPDATE:]

I am trying to make the Let's Encrypt plugin work with HAProxy and I ran into this situation:

When I start HAProxy from the UI it fails to start.
I SSH-ed into the firewall and tried to start it manually with:

Code Select Expand

/usr/local/etc/rc.d/haproxy start

Then I got this alert:

Code Select Expand

Starting frontend *****: cannot bind socket *******

From searching online, I found these possible solutions (although those seem to be for Linux, not BSD):

https://stackoverflow.com/questions/34793885/haproxy-cannot-bind-socket-0-0-0-08888
https://discourse.haproxy.org/t/solved-cannot-bind-socket/3180/4

They essentially suggest using this:

Code Select Expand

setsebool -P haproxy_connect_any=1

So, my questions are:

• Why doesn't this detailed error, that I get in the console, appear on the management GUI? I only get a generic message saying that HAProxy failed to start. I have searched the log level settings, but nothing seems to provide more detailed information. Am I missing something?
• Is this suggested solution the proper one? I would expect that the HAProxy plugin should work out of the box, without manual fiddling into the OS. Is there any chance I caused the misconfiguration myself somehow? E.g. using the "System>Firmware>Updates" option?

UPDATE:
On top of this, it turns out that 'setsebool' is an unknown command in my OpnSense box. Do I have to install something or use a different command?

Forget it.
I found it!

/usr/local/etc/haproxy.conf

I am new to OpnSense and I had a problem with HaProxy recently.
I posted my question to the HaProxy forum ( https://discourse.haproxy.org/t/http-response-set-header-with-condition-not-working/3108/6?u=gazelle3574 ), but they ask for the HAProxy configuration file.
Any idea how I can get it?
I have connected with SSH on the firewall and searched for quite some time (over half an hour), but I found nothing. Only some HAProxy-related files, but not the config.  :'(

The weird part is that I use this :

Code Select Expand

find / -name '*.cfg'
...and I only find some Python-related files and nothing else.

Any ideas?

Thanks in advance.