Messages

Proxmox host: ext4
VM storage: LVM-thin
OPNSense: ZFS

[Zenarmor ON, 15 minutes]

Since no one at home right now, the loading of my router is low.
My test is as below:

Zenarmor ON, 15 minutes
127,049,983>127,051,329
=1346 data units
≈688.6MB written in 15 minutes
≈66GB/day

Zenarmor OFF, 15 minutes
127,052,544>127,052,839
=295 data units
≈150.9MB written in 15 minutes
≈14GB/day

Environment:
Base OS: Proxmox 9.2.11
Kernel Version: Linux 7.0.14-15-pve
SSD (hardware): Samsung SSD 970 EVO Plus 500GB
OPNsense VM: (Disk setting only) LVM-Thin, 32GB, Cache=Default (no Cache), Discard=on, IO thread=on, SSD emulation=on
OPNsense: OPNsense 26.7.3_11-amd64 FreeBSD 15.1-RELEASE-p3
Zenarmor Engine: 2.6.2
Zenarmor Database: 2.0.26071708
Reporting backend: Local SQLite 3.53.4,1

Can you try this?
https://forum.opnsense.org/index.php?topic=38989.0

Just tried adding

Code Select Expand

exception-policy: ignore
to
/usr/local/opnsense/service/templates/OPNsense/IDS/custom.yaml

no more drop for VoIP now. Thanks.
I will try again 12 hours later to confirm it won't drop anymore.

thanks for reply , but this not what i mean

i mean to include the shaping inside the firewall rule

FW policy:

Allow - From source -- to desintation -going to specific shapping queue

give them priority inside queue High Priority which i defined

i am not sure if i am not clear in my question

https://docs.opnsense.org/manual/how-tos/shaper_prioritize_using_queues.html
The rules inside traffic shaping already doing the same thing.

Hello all,

Does anyone know a way to factory reset only HAProxy? There is an old configuration in there and i stopped using it for a while. Now i cannot remove the old config and rules and such. It's giving me errors. Starting over with HAProxy seems like the easiest way to solve it.

Thanks,
Misja

If you want to remove all rules, frontends and backends
You need to remove them with correct order (as you cannot remove one item when other is still using it):
Frontends->Rules->Conditions->Map Files->Backends->Real Servers

(I don't know any other way to reset the configuration)

Thank you fir this nice guide i followed to replace nginx reverse proxy. i left my
nexcloud server untouched.
After following the guide, i've got "too many redirects" error.
i've looking around for a possible solution... but i don't understand what is wrong
i'm using cloudflare and this it's apache configuration

Code Select Expand


<VirtualHost *:80>
    DocumentRoot "/usr/local/www/nextcloud"
    ServerName nextcloud.mysite.com
<FilesMatch \.php$>
        SetHandler "proxy:fcgi://127.0.0.1:9000/"
    </FilesMatch>
    DirectoryIndex /index.php index.php
</VirtualHost>


config.php

Code Select Expand



{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "10.0.0.48",
            "nextcloud.mysite.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "25.0.4.1",
        "overwrite.cli.url": "https:\/\/nextcloud.mysite.com",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwritehost": "nextcloud.mysite.com",
        "overwriteprotocol": "https",
        "overwritecondaddr": "^10\\.0\\.0\\.1$",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "maintenance": false,
        "theme": "",
        "loglevel": 2,
        "updater.release.channel": "stable",
        "default_phone_region": "IT",
        "app_install_overwrite": [],
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [
            "admin"
        ],
        "twofactor_enforced_excluded_groups": [],
        "data-fingerprint": "d1c023081e0c9b662bc8049cf295c443"
    }
}




haproxy.conf

Code Select Expand



global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_frontend (listening on 0.0.0.0:80. 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options

# Frontend: 1_HTTP_frontend (Listening on 127.4.4.3:80)
frontend 1_HTTP_frontend
    bind 127.4.4.3:80 name 127.4.4.3:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    # ACL: NoSSL_condition
    acl acl_64188d5dce2390.01132494 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_64188d5dce2390.01132494

# Frontend: 1_HTTPS_frontend (Lisening on 127.4.4.3:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.4.4.3:443 name 127.4.4.3:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/64189270e357f4.63771565.certlist
    mode http
    option http-keep-alive
    option forwardfor
    timeout client 15m

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/64188dd26c8986.37023969.txt)]

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server SSL_server 127.4.4.3 send-proxy-v2 check-send-proxy

# Backend: Nextcloud_backend ()
backend Nextcloud_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server Nextcloud_server 10.0.0.48:80 ssl verify none





Have you tried removing the value of the following items?
"overwrite.cli.url"
"overwritehost"
"overwriteprotocol"
"overwritecondaddr"

Since SNI is handled by HAProxy already. I don't think they are necessary.
And the issue might caused by this too.
(HAProxy tried to reach 10.0.0.48:80, nextcloud response to browser it need to "overwrite"(redirect) to https://nextcloud.mysite.com, but browser is already accessing with this path. So the browser don't know what should be redirect (or having dead loop for the worst case) and give you "too many redirects" error.

Which URL do you use to access them from inside?

How do you access them directly (ip:port) full URL?

Bitwarden is accesses via Bitwarden.mydomain.com for internal

(it works) and external (can't reach server)
HA is accessed internally from 192.168.1.106:8123 (works) and external via homeassistant.mydomain.com (can't reach server).

There are some settings for HA to configure proxy that I think I need to set up, but since Bitwarden doesn't work it's not the complete solution to my problem.


From https://www.home-assistant.io/integrations/http/

use_x_forwarded_for boolean (optional, default: false)
Enable parsing of the X-Forwarded-For header, passing on the client's correct IP address in proxied setups. You must also whitelist trusted proxies using the trusted_proxies setting for this to work. Non-whitelisted requests with this header will be considered IP spoofing attacks, and the header will, therefore, be ignored.

trusted_proxies string | list (optional)
List of trusted proxies, consisting of IP addresses or networks, that are allowed to set the X-Forwarded-For header. This is required when using use_x_forwarded_for because all requests to Home Assistant, regardless of source, will arrive from the reverse proxy IP address. Therefore in a reverse proxy scenario, this option should be set with extreme care. If the immediate upstream proxy is not in the list, the request will be rejected. If any other intermediate proxy is not in the list, the first untrusted proxy will be considered the client.

I'm using suricata IPS/IDS and tried disabling these, but no change. Using blocklists in Unbound DNS but that should not interfere.

I'm out of ideas. I will review everything again when I get home later today.

# public access subdomains
bit BITWARDEN_backend
home HA_backend

From the map file you provided, you should be accessing your Bitwarden with bit.mydomain.com in local network
If you failed with bit.mydomain.com locally, check your Unbound override to see if you have override bit.mydomain.com to the IP that TCP frontend listening to (That means, LAN or VLAN gateways or Loopback IPs)

Until you can access internally with bit.mydomain.com (not any other path), then you check rather you set your WAN port forward correctly.

BTW. I have Bitwarden running in my home, can access from both LAN and WAN
Suricata and Zenarmor and dns blocklist are on for my network. They won't block your Bitwarden.
The following list is related to your issue
1. (Scope: both internal and external)Rather HAProxy setup correctly (If you follow the guide without missing any point, that should be correct)
2. (Scope: Internally) Internal DNS override bit.mydomain.com to your TCP fontend
3. (Scope: Externally) WAN port forward at least TCP 443 to TCP frontend
4. (Scope: Externally) DDNS updated bit.mydomain.com to your WAN address

[Bunch - you explain very clearly and extensively (big plus for you).]

Bunch - you explain very clearly and extensively (big plus for you).   :)

Once I digest what you wrote, I'll come back to the forum.

In the case of VPN, I will create a separate topic, but it has a short question: do you need to buy a VPS in the cloud to change the IP?

poscriptum: I was looking for someone like you. You have the patience to write and can explain clearly.

I don't think you need to do something like this.
The chance of VPS servers being attack is not lower than a home network. Thus, it won't bring you extra security with VPS as relay server.
If you are having dynamic IP from ISP, and you afraid you cannot access your VPN after IP change. You only need to setup DDNS.

[thank you very much for the clear hint]

Hi Bunch, thank you very much for the clear hint.  :)
Everything worked!

I am new to OPNsense, can I ask you for help in the future?
I am very interested in the topic: VPN -> OpenVPN: (like IP masquerading). Can I ask (you) questions on this topic?
I am also interested in topics related to Firewall -> Rules (when and how to use it)?

postscript:
I'm 16 and just learning about this topic.  :)

For VPN related topic. I only use IPSec and Wireguard now, and it is better to create another topic in VPN board


For firewall rules. Since I don't know where should be posted (maybe General discussion?), I will keep answering here. WAN interface will block all incoming connections by default (except those sessions that start by your clients)

When you open a port in WAN interface (for example you open a port for web hosting), there is a chance that a new vulnerability for the protocol being found and you're attacked before hotfix applied (zero-day attack). You want to reduce the chance, so you will limit the IPs that can reach to your services by using WAN rules.
For example, the list of IP: https://rules.emergingthreats.net/blockrules/compromised-ips.txt
This list of IP are found to be compromised, you can create a new alias in Firewall: Aliases
Name: IPBlocklists
Type: URL Table (IPs)
Refresh Frequency: Any time you like, for example every 12 hours
Content: https://rules.emergingthreats.net/blockrules/compromised-ips.txt
(You may add more if you want)
Description: IP Blocklists

Then you create a rule in WAN:
Action: Block
Quick: true
Interface: WAN
Direction: in
TCP/IP Version: IPv4 (since I only have IPv4 connection from ISP and the list is IPv4 too)
Protocol: any
Source: IPBlocklists
Destination: any
Log: true (if you want to see how many connections are blocked by this rule)
Category: Block using Blocklist
(This is the name that shows in firewall log if you enabled log)
Description: Block using Blocklist

Then move the new block rule on top of the rule port opening rule.

You may also reject LAN IPs to access those IPs.
Action: Reject
Quick: true
Interface: LAN
Direction: in
TCP/IP Version: IPv4
Protocol: any
Source: any
Destination: IPBlocklists
Log: true (if you want to see how many connections are blocked by this rule)
Category: Reject using Blocklist
(This is the name that shows in firewall log if you enabled log)
Description: Reject using Blocklist

The follow are the lists that I used, some of them might be duplicated, and it could be too aggressive in some cases

Code Select Expand

https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset
https://cinsarmy.com/list/ci-badguys.txt
https://rules.emergingthreats.net/blockrules/compromised-ips.txt
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://sslbl.abuse.ch/blacklist/sslipblacklist.txt
https://feodotracker.abuse.ch/downloads/ipblocklist.txt
https://lists.blocklist.de/lists/all.txt
https://reputation.alienvault.com/reputation.generic
https://www.dan.me.uk/torlist/
https://www.spamhaus.org/drop/drop.txt

Another example is to limit the countries that can access to your service (VPN for example)
My family will only travel between HK, UK and AUS, and I only allow them to access VPN service.
You may subscribe GeoIP according the opnsense manual
Then create a alias using geo IP and select all countries, except HK, UK and AUS.
Create a firewall rule that block the GeoIP alias like the blocklist above (you can limit the destination port to your VPN port if you think that's too aggressive) and place it before the rule that opening port for VPN.
*Don't create LAN rule for GeoIP

There are some advanced usage using port forward etc.
You may take a look in Tutorials and FAQs

Hello everyone.

Three days ago I installed OPNsense 22.7-amd64 and would like to make changes to the squid.conf file.
I read that you can do this in the squid directory: /usr/local/etc/squid/squid.conf

I have a problem, I don't know how to do it ;-(
Can my friends explain it to me like a child?

What should I do next. How to get to this path - /usr/local/opnsense/service/templates/ - where to find it and make changes to the squid.conf file.

Please give me a hint.

For OPNSense setting
System: Settings: Administration: Secure Shell
Secure Shell Server: true
Login Group: wheel, admins
Root Login: true
Authentication Method: true
SSH port: 22
Listen Interfaces: LAN

Download Putty and use it to connect to your router
If you familiar with vi editor in linux, after login with putty

Code Select Expand

sudo vi /usr/local/etc/squid/squid.conf
If you are not familiar with vi editor, you may install nano and use it as editor

Code Select Expand

pkg install nano

Code Select Expand

sudo nano /usr/local/etc/squid/squid.conf

If you want to edit /usr/local/opnsense/service/templates/squid.conf
Simply change the path above

Have you added ddns in System: Settings: Administration: Alternate Hostnames?

Thank you for taking the time to reply to my thread. Will try this on the weekend and reply back.
I have another question in the meantime, on the reply by alh he states on number 4

4. HAProxy plugin: Create "condition" (enter name ["myservice_sni"], condition type is "SNI TLS extension matches (TCP request content inspection)" with value "myservice.example.com" or whatever your FQDN is)

I dont really understand how this one works. is the myservice.example.com supposed to be the traefik dashboard domain? or can i put a wildcard certificate for it, something like *.mydomain.example?

Thank you once again.

No. There aren't any condition accept partial regex in haproxy. If you want to pass the whole wildcard domain to Traefik
Change the condition to "SNI TLS extension contains (TCP request content inspection)" or "SNI TLS extension ends with (TCP request content inspection)"
For example
SNI TLS extension contains (TCP request content inspection)
.example.com

The whole thing is the usual practice to get HAProxy working with SNI in TCP layer
Check "Restrict clients to SSH only" here
And the guide is how he managed to put those commands in specific order

You have set traefik_backend to use Proxy Protocol
Have to set traefik to accept proxy protocol?
If not, either you find a way setting traefik to accept proxy protocol.
Or, you remove the Proxy Protocol config in traefik_backend. (But traefik and all its backends won't get correct IP info)

@franco what is a resolution? in the log to unbound:
blocklist download : unable to download file from https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/porn/hosts (error : HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /StevenBlack/hosts/master/alternates/porn/hosts (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x80261cc10>: Failed to establish a new connection: [Errno 8] Name does not resolve'))).

Thanks ;)

It doesn't matter what list others using, when you can't even resolve those listed in stock.
The possible cause is there is no upstream server set for your OPNSense.

You need to set
either
using ISP DNS server as upstream server of OPNSense (System: Settings: General, Allow DNS server list to be overridden by DHCP/PPP on WAN. Services: Unbound DNS: Query Forwarding, Use System Nameservers)
or
using other servers as upstream server (System: Settings: General, add 1.1.1.1 if you want to use cloudflare. Services: Unbound DNS: Query Forwarding, Use System Nameservers)

If you confirmed you have set either one of it.
Check Interfaces: Overview: WAN interface to see what DNS servers are pushed by your ISP and try
Interfaces: Diagnostics: DNS Lookup
Hostname or IP: raw.githubusercontent.com
Server: one of the listed DNS server you found in Overview

If the name cannot be resolved by your ISP, it can be blocked by your ISP and your ISP redirected all DNS traffic to her own server. DOT would be solution for that case

I've tried to make a Port forward rule (see attached screenshot), but I'm not sure I'm doing it right.

It doesn't make a difference on my phone, the lan ip is still shown.

Change the destination to any

Firewall: NAT: Port Forward, Create new rule
Interface: Home (assume your wireless access point is in Home interface)
TCP/IP Version: IPv4
Protocol: UDP
Destination: any
Destination port range: DNS to DNS
Redirect target IP: 127.0.0.1
Redirect target port: 53

If that still doesn't work, your phone is probably using DNS over TLS or DNS over HTTP.
Which cannot be redirected.