[SOLVED] HAPRoxy bug & missing feature (1.1)

[manus]

- In Global Parameters, Custom options have no effect (nothing in /usr/local/etc/haproxy.conf).
- Add Custom options in Default Parameters and Statistics Configuration.
- Add default-server entry for backend and server must be after default-server in final file (server is recommended to be last entry in all case):

Code: [Select]

default-server port 21 inter 10s downinter 15s rise 3 fall 2 slowstart 60s weight 100- Unable to create server without port (to use frontend port):

Code: [Select]

server MyServerName 10.10.5.1: check- Unable to create frontend with port range:

Code: [Select]

bind 31.9.36.15:20-21
bind 31.9.36.15:49000-49500- If you add a frontend without ssl&certificate after one with ssl&certificate, the frontend without ssl&certificate got a certificate and ssl option:

Code: [Select]

bind 31.9.36.15:2222 name 31.9.36.15:2222 ssl   crt /var/etc/haproxy/ssl/5752ccd0803c0.pem
instead of:

Code: [Select]

bind 31.9.36.15:2222 name 31.9.36.15:2222- Service unable to run with this in global ("Some configuration options require full privileges, so global.uid cannot be changed."):

Code: [Select]

uid                         80

[manus]

To get Global Parameters/Custom options working, add this at the end of GLOBAL section in /usr/local/opnsense/service/templates/OPNsense/HAProxy/haproxy.conf :

Code: [Select]

{% if OPNsense.HAProxy.general.tuning.customOptions|default("") != "" %}
    # WARNING: pass through options below this line
{%   for customOpt in OPNsense.HAProxy.general.tuning.customOptions.split("\n") %}
    {{customOpt}}
{%   endfor %}
{% endif %}
Fix frontend without ssl&certificate after one with ssl&certificate:

Code: [Select]

{# ############################### #}
{#             FRONTENDS           #}
{# ############################### #}

{% if helpers.exists('OPNsense.HAProxy.frontends') %}
{%   for frontend in helpers.toList('OPNsense.HAProxy.frontends.frontend') %}
{%     if frontend.enabled == '1' %}
# Frontend: {{frontend.name}} ({{frontend.description}})
frontend {{frontend.name}}
{%       if frontend.ssl_enabled == '1' %}
{#         # collect ssl certs (if configured) #}
{%         if frontend.ssl_certificates|default("") != "" %}
{%           set ssl_certs = [] %}
{%           for cert in frontend.ssl_certificates.split(",") %}
{%             do ssl_certs.append('crt /var/etc/haproxy/ssl/' ~ cert ~ '.pem') %}
{%           endfor %}
{%         endif %}
{#         # advanced ssl options #}
{%         if frontend.ssl_customOptions|default("") != "" %}
{#           # add a space to separate it from other ssl params #}
{%           set ssl_options = frontend.ssl_customOptions ~ ' ' %}
{%         endif %}
{%       endif %}
{#       # bind/listen configuration #}
{%       if frontend.bind|default("") != "" %}
{%         for bind in frontend.bind.split(",") %}
    bind {{bind}} name {{bind}} {% if frontend.ssl_enabled == '1' and ssl_certs|default("") != "" %}ssl {{ ssl_options }}{{ssl_certs|join(' ')}}{%
endif %}

{%         endfor %}
{%       endif %}
Put server at end of backend (remove old '{%       for server in backend.linkedServers.split(",") %}...{%       endfor %}' before adding this code at end):

Code: [Select]

{%       for server in backend.linkedServers.split(",") %}
{%         set server_data = helpers.getUUID(server) %}
{#         # collect optional server parameters #}
{%         set server_options = [] %}
{#       if# check if health check is enabled #}("") != "" %}
{%         if healthcheck_enabled == '1' %}
{%           do server_options.append('check') %}
{%           do server_options.append('inter ' ~ server_data.checkInterval) %}
{#           # add all additions from healthchecks here #}
{%           do server_options.append(healthcheck_additions|join(' ')) if healthcheck_additions.length != '0' %}
{%         endif %}
{#         # server weight #}
{%         do server_options.append('weight ' ~ server_data.weight) if server_data.weight|default("") != "" %}
{#         # server role/mode #}
{%         do server_options.append(server_data.mode) if server_data.mode|default("") != "active" %}
    server {{server_data.name}} {{server_data.address}}:{{server_data.port}} {{server_options|join(' ')}}
{%       endfor %}

{%     else %}
# Backend (DISABLED): {{backend.description}}

{%     endif %}
{%   endfor %}
{% endif %}
To add default-server parameter:
In /usr/local/opnsense/service/templates/OPNsense/HAProxy/haproxy.conf before new '{%       for server in backend.linkedServers.split(",") %}':

Code: [Select]

{%       if backend.tuning_defaultserver|default("") != "" %}
    default-server {{backend.tuning_defaultserver}}
{%       endif %}In /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.xml in backend section after customOptions:

Code: [Select]

                <tuning_defaultserver type="TextField">
                    <Required>N</Required>
                </tuning_defaultserver>In /usr/local/opnsense/mvc/app/controllers/OPNsense/HAProxy/forms/dialogBackend.xml after backend.customOptions section:

Code: [Select]

    <field>
        <id>backend.tuning_defaultserver</id>
        <label>Default for server</label>
        <type>text</type>
        <help><![CDATA[Default option for all server entries.]]></help>
    </field>
Add frontend range port:

Code: [Select]

                <bind type="CSVListField">
                    <Required>Y</Required>
                    <multiple>Y</multiple>
                    <!-- <default>localhost:8080</default> -->
                    <mask>/^((([0-9a-zA-Z._\-\*]+:[0-9]+(-[0-9]+)?)([,]){0,1}))*/u</mask>
                    <ChangeCase>lower</ChangeCase>
                    <ValidationMessage>Please provide a valid listen address, i.e. 127.0.0.1:8080 or www.example.com:443. Port
range as 1210-1220.</ValidationMessage>
                </bind>

[franco]

Hi Manus,

Thanks for your analysis! I've added a ticket on GitHub and assigned it to the maintainer.

https://github.com/opnsense/plugins/issues/17


Cheers,
Franco

[manus]

We need to enable IPFW to use some custom rules for HAProxy, for exemple I need that for my backend server:

Code: [Select]

table 66 flush
table 66 add 10.10.12.1/32
table 66 add 10.10.12.2/32
table 66 add 10.10.12.3/32
table 66 add 10.10.12.4/32
table 66 list
list
add 10 fwd localhost tcp from table(66) 4480 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 22 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 21 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 49000-49500 to any in recv vmx1
list
So, I modified /usr/local/opnsense/service/templates/OPNsense/IPFW/rc.conf.d:

Code: [Select]

{% if (helpers.exists('OPNsense.HAProxy.general') and OPNsense.HAProxy.general.enabled|default("0") == "1") %}
{%     set haproxy_enable = 1 %}
{% endif %}
firewall_enable="{% if shapers or cp_zones or haproxy_enable %}YES{% else %}NO{% endif %}"
And /usr/local/etc/rc.ipfw:

Code: [Select]

# reload ipfw rules
/sbin/ipfw -f /usr/local/etc/ipfw.rules
if [ -f /usr/local/etc/ipfw_custom.rules ]; then
    /sbin/ipfw -f /usr/local/etc/ipfw_custom.rules
fi
And add all my rules in /usr/local/etc/ipfw_custom.rules

[manus]

See second post for fix (2016-06-04 10:08:18 PM)

[manus]

Please see the pull request for all fix and more:
https://github.com/opnsense/plugins/pull/19

[fraenki]

We need to enable IPFW to use some custom rules for HAProxy, for exemple I need that for my backend server: [...]

So, I modified /usr/local/opnsense/service/templates/OPNsense/IPFW/rc.conf.d: [...]

And /usr/local/etc/rc.ipfw: [...]

Thanks. Currenty plugins (like HAProxy) can't inject new firewall rules... but this feature is planned. Maybe this isn't enough for your use-case, so feel free to open another feature request.

EDIT: Have you tried to add firewall rules for this through the GUI? I think this should be possible, I wonder why IPFW should be required for this...

[manus]

For some use, we need ipfw rules. Personnaly I need it for transparent proxy, send client IP to backend and it required to have the HAProxy as default gateway on backend server, i.e.:
ipfw table 66 add 10.12.19.1/32
ipfw table 66 add 10.12.19.2/32
ipfw table 66 add 10.12.19.3/32
ipfw table 66 add 10.12.19.4/32
ipfw add 10 fwd localhost tcp from 'table(66)' 3380 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 22 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 21 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 48500-48700 to any in recv vmx1

[franco]

The fixes have made it into version 1.3, thanks again!

As long as there is no traffic shaping or captive portal on the server IPFW should be OK to be configured independently. Is there something interfering with this, e.g. empty rules file after reboot?

[manus]

If no one needs the IPFW feature, we can stay with 1.3, I will patch my OpnSense each time, it's not a big deal.
So, maybe we can wait for other users with this need before implement it.

[franco]

Glad to hear. I'm asking though to make sure that this maybe does not need patching through firmware upgrades. We can e.g. successfully run additional services without adding them to the GUI and without having to fix them after every firmware update by using standard FreeBSD setups through the rc(8) system.

[franco]

Hrm, 1.3 will have all things but the non-chroot feature. If it's still needed we should put it back in 1.4 as a non-default. I just noticed so I thought I let you know. We were rearranging things with chroot and could not come up with a use case that requires it. Can you please explain?