Messages

Since 4 or 5 minor release, we have a 100% CPU on HAProxy after some time (1-2 months).
Only a reboot solve the problem.
Anybody with this problem too?

When HAProxy will be updated to 1.7.x? or at minima to last 1.6 (1.6.12)?

If no one needs the IPFW feature, we can stay with 1.3, I will patch my OpnSense each time, it's not a big deal.
So, maybe we can wait for other users with this need before implement it.

For some use, we need ipfw rules. Personnaly I need it for transparent proxy, send client IP to backend and it required to have the HAProxy as default gateway on backend server, i.e.:
ipfw table 66 add 10.12.19.1/32
ipfw table 66 add 10.12.19.2/32
ipfw table 66 add 10.12.19.3/32
ipfw table 66 add 10.12.19.4/32
ipfw add 10 fwd localhost tcp from 'table(66)' 3380 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 22 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 21 to any in recv vmx1
ipfw add 10 fwd localhost tcp from 'table(66)' 48500-48700 to any in recv vmx1

Please see the pull request for all fix and more:
https://github.com/opnsense/plugins/pull/19

See second post for fix (2016-06-04 10:08:18 PM)

We need to enable IPFW to use some custom rules for HAProxy, for exemple I need that for my backend server:

Code Select Expand

table 66 flush
table 66 add 10.10.12.1/32
table 66 add 10.10.12.2/32
table 66 add 10.10.12.3/32
table 66 add 10.10.12.4/32
table 66 list
list
add 10 fwd localhost tcp from table(66) 4480 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 22 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 21 to any in recv vmx1
add 11 fwd localhost tcp from table(66) 49000-49500 to any in recv vmx1
list

So, I modified /usr/local/opnsense/service/templates/OPNsense/IPFW/rc.conf.d:

Code Select Expand

{% if (helpers.exists('OPNsense.HAProxy.general') and OPNsense.HAProxy.general.enabled|default("0") == "1") %}
{%     set haproxy_enable = 1 %}
{% endif %}
firewall_enable="{% if shapers or cp_zones or haproxy_enable %}YES{% else %}NO{% endif %}"

And /usr/local/etc/rc.ipfw:

Code Select Expand

# reload ipfw rules
/sbin/ipfw -f /usr/local/etc/ipfw.rules
if [ -f /usr/local/etc/ipfw_custom.rules ]; then
    /sbin/ipfw -f /usr/local/etc/ipfw_custom.rules
fi

And add all my rules in /usr/local/etc/ipfw_custom.rules

To get Global Parameters/Custom options working, add this at the end of GLOBAL section in /usr/local/opnsense/service/templates/OPNsense/HAProxy/haproxy.conf :

Code Select Expand

{% if OPNsense.HAProxy.general.tuning.customOptions|default("") != "" %}
    # WARNING: pass through options below this line
{%   for customOpt in OPNsense.HAProxy.general.tuning.customOptions.split("\n") %}
    {{customOpt}}
{%   endfor %}
{% endif %}

Fix frontend without ssl&certificate after one with ssl&certificate:

Code Select Expand

{# ############################### #}
{#             FRONTENDS           #}
{# ############################### #}

{% if helpers.exists('OPNsense.HAProxy.frontends') %}
{%   for frontend in helpers.toList('OPNsense.HAProxy.frontends.frontend') %}
{%     if frontend.enabled == '1' %}
# Frontend: {{frontend.name}} ({{frontend.description}})
frontend {{frontend.name}}
{%       if frontend.ssl_enabled == '1' %}
{#         # collect ssl certs (if configured) #}
{%         if frontend.ssl_certificates|default("") != "" %}
{%           set ssl_certs = [] %}
{%           for cert in frontend.ssl_certificates.split(",") %}
{%             do ssl_certs.append('crt /var/etc/haproxy/ssl/' ~ cert ~ '.pem') %}
{%           endfor %}
{%         endif %}
{#         # advanced ssl options #}
{%         if frontend.ssl_customOptions|default("") != "" %}
{#           # add a space to separate it from other ssl params #}
{%           set ssl_options = frontend.ssl_customOptions ~ ' ' %}
{%         endif %}
{%       endif %}
{#       # bind/listen configuration #}
{%       if frontend.bind|default("") != "" %}
{%         for bind in frontend.bind.split(",") %}
    bind {{bind}} name {{bind}} {% if frontend.ssl_enabled == '1' and ssl_certs|default("") != "" %}ssl {{ ssl_options }}{{ssl_certs|join(' ')}}{%
endif %}

{%         endfor %}
{%       endif %}

Put server at end of backend (remove old '{%       for server in backend.linkedServers.split(",") %}...{%       endfor %}' before adding this code at end):

Code Select Expand

{%       for server in backend.linkedServers.split(",") %}
{%         set server_data = helpers.getUUID(server) %}
{#         # collect optional server parameters #}
{%         set server_options = [] %}
{#       if# check if health check is enabled #}("") != "" %}
{%         if healthcheck_enabled == '1' %}
{%           do server_options.append('check') %}
{%           do server_options.append('inter ' ~ server_data.checkInterval) %}
{#           # add all additions from healthchecks here #}
{%           do server_options.append(healthcheck_additions|join(' ')) if healthcheck_additions.length != '0' %}
{%         endif %}
{#         # server weight #}
{%         do server_options.append('weight ' ~ server_data.weight) if server_data.weight|default("") != "" %}
{#         # server role/mode #}
{%         do server_options.append(server_data.mode) if server_data.mode|default("") != "active" %}
    server {{server_data.name}} {{server_data.address}}:{{server_data.port}} {{server_options|join(' ')}}
{%       endfor %}

{%     else %}
# Backend (DISABLED): {{backend.description}}

{%     endif %}
{%   endfor %}
{% endif %}

To add default-server parameter:
In /usr/local/opnsense/service/templates/OPNsense/HAProxy/haproxy.conf before new '{%       for server in backend.linkedServers.split(",") %}':

Code Select Expand

{%       if backend.tuning_defaultserver|default("") != "" %}
    default-server {{backend.tuning_defaultserver}}
{%       endif %}
In /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.xml in backend section after customOptions:

Code Select Expand

                <tuning_defaultserver type="TextField">
                    <Required>N</Required>
                </tuning_defaultserver>
In /usr/local/opnsense/mvc/app/controllers/OPNsense/HAProxy/forms/dialogBackend.xml after backend.customOptions section:

Code Select Expand

    <field>
        <id>backend.tuning_defaultserver</id>
        <label>Default for server</label>
        <type>text</type>
        <help><![CDATA[Default option for all server entries.]]></help>
    </field>

Add frontend range port:

Code Select Expand

                <bind type="CSVListField">
                    <Required>Y</Required>
                    <multiple>Y</multiple>
                    <!-- <default>localhost:8080</default> -->
                    <mask>/^((([0-9a-zA-Z._\-\*]+:[0-9]+(-[0-9]+)?)([,]){0,1}))*/u</mask>
                    <ChangeCase>lower</ChangeCase>
                    <ValidationMessage>Please provide a valid listen address, i.e. 127.0.0.1:8080 or www.example.com:443. Port
range as 1210-1220.</ValidationMessage>
                </bind>

- In Global Parameters, Custom options have no effect (nothing in /usr/local/etc/haproxy.conf).
- Add Custom options in Default Parameters and Statistics Configuration.
- Add default-server entry for backend and server must be after default-server in final file (server is recommended to be last entry in all case):

Code Select Expand

default-server port 21 inter 10s downinter 15s rise 3 fall 2 slowstart 60s weight 100
- Unable to create server without port (to use frontend port):

Code Select Expand

server MyServerName 10.10.5.1: check
- Unable to create frontend with port range:

Code Select Expand

bind 31.9.36.15:20-21
bind 31.9.36.15:49000-49500
- If you add a frontend without ssl&certificate after one with ssl&certificate, the frontend without ssl&certificate got a certificate and ssl option:

Code Select Expand

bind 31.9.36.15:2222 name 31.9.36.15:2222 ssl   crt /var/etc/haproxy/ssl/5752ccd0803c0.pem

instead of:

Code Select Expand

bind 31.9.36.15:2222 name 31.9.36.15:2222
- Service unable to run with this in global ("Some configuration options require full privileges, so global.uid cannot be changed."):

Code Select Expand

uid                         80

I change /usr/local/etc/rc.ipfw:

Code Select Expand

# reload ipfw rules
/sbin/ipfw -f /usr/local/etc/ipfw.rules
if [ -f /usr/local/etc/ipfw_custom.rules ]; then
    /sbin/ipfw -f /usr/local/etc/ipfw_custom.rules
fi


/usr/local/opnsense/service/templates/OPNsense/IPFW/rc.conf.d:

Code Select Expand

firewall_enable="{% if shapers or cp_zones %}YES{% else %}YES{% endif %}"

Could you add an option in interface or through a config file to enable IPFW.
Currently we can't use default rc file because /usr/local/opnsense/service/templates/OPNsense/IPFW/rc.conf.d rewrite the firewall_enable.
So to use IPFW, I need to add a traffic shaper rules&co to use IPFW or modify previous file.
The best will be to get :
- an option in interface to enable IPFW (firewall_enable)
- an option to add ipfw_load="YES" in /boot/loader.conf or other way to load kernel module at boot
- a field to give a full path to a file with rules to execute after:
# reload ipfw rules
/sbin/ipfw -f /usr/local/etc/ipfw.rules

Why? because I use a compiled HAProxy in transparent mode. But some can use it for others things.