Messages

Hi

I have a OPNsene router behind him is a server with qBittorrent-nox 4.5.5 as client.

The Upload speed is proper but the download speed breaks down after seconds and if it's highspeed.

How I can fix this?

Ok. I changed OPNsene Port to 222 and SSH_frontend to 22.

Now I can access all my clients over haproxy with ssh,ipv4 and port 22.

Thanks for the Input.

Thx.

I found that on pfsense TCP widows size not be touched by the firewall. (Nothing found for OPNsense)

https://support.pfsense.narkive.com/ESAuH5Oy/tcp-tuning-for-pfsense

I will sniff with wireshark the TCP window size of the connection.

https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#intel-ix-4-cards

Is this also guilty for ixl cards? I can't find anything for ixl cards only ix.

I have no Cloud access with 25 Gbps.

Code Select Expand


ifconfig | grep ixl1

ixl1: flags=8a63<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: WAN
options=4800028<VLAN_MTU,JUMBO_MTU,NOMAP>
ether 64:9d:99:b1:b4:ae
inet XXX netmask 0xffffff00 broadcast XXX
inet6 XXX::669d:99ff:feb1:b4ae%ixl1 prefixlen 64 scopeid 0x4
inet6 XXX:168:XXX:33:669d:99ff:feb1:b4ae prefixlen 64 autoconf
inet6 XXX:168:XXX:33::26 prefixlen 128
media: Ethernet autoselect (25GBase-LR <full-duplex>)
status: active
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>


Code Select Expand


sysctl hw.model hw.machine hw.ncpu

hw.model: Intel(R) Xeon(R) W-3223 CPU @ 3.50GHz
hw.machine: amd64
hw.ncpu: 16


Yes.

I have a Intel NIC and should not have any isssues.
Xeon CPU loads go never over 30%
I believe there is one stream.

Jumboframes aren't supported by ISP.

Xeon W-? a eightcore CPU with enough power.

Hi

I have a ISP Connection of 25 Gbps and a NIC with 2x25 Gbps. When I lunch Ookla CLI Speedtest I get 10 Gbps in Up/Download but no ~20 Gbps. The Server ID of ooklaserver is 43030.

Are 25 Gbps possible with OPNsense...

Code Select Expand


#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 1000s
    timeout connect 1000s
    timeout server 1000s
    retries 3
    default-server init-addr libc,last
    default-server maxconn 5000

# autogenerated entries for ACLs

# userlists generated from groups
userlist Allowedusers
    user joel insecure-password XXX
    user mopidy insecure-password XXX
    # NOTE: UserlistAddUsers called with empty group data


# autogenerated entries for config in backends/frontends
userlist list_6245eeb66d3ab2.08976803
    # Origin: MOPIDY_backend
    user mopidy insecure-password XXX
    user joel insecure-password XXX
    # WARNING: skipping duplicate username (mopidy)


# autogenerated entries for stats




# Frontend: SNI_frontend (Listening on http&https)
frontend SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind :::80 name :::80
    bind :::443 name :::443
    mode tcp
    default_backend SSL_backend
    # tuning options
    timeout client 1000s

    # logging options

# Frontend: HTTP_frontend (Listening 127.0.0.1:80)
frontend HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    bind [::1]:80 name [::1]:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 1000s

    # logging options
    # ACL: NoSSL_condition
    acl acl_621d0b77c74989.24704837 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_621d0b77c74989.24704837

# Frontend: HTTPS_frontend (Listinging on 127.0.0.1:443)
frontend HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/621d11c7cad951.61400293.certlist
    bind [::1]:443 name [::1]:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/621d11c7cad951.61400293.certlist
    mode http
    option http-keep-alive
    default_backend WEBSERVER_backend
    option forwardfor
    # tuning options
    timeout client 15m

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_map-rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/621d0c7054ddb7.46420139.txt)]
    # WARNING: pass through options below this line
      # Matrix client traffic
      acl matrix-host hdr(host) -i chat.XXX.ch chat.XXX.ch:443
      acl matrix-path path_beg /_matrix
      acl matrix-path path_beg /_synapse/client
   
      use_backend MATRIX_backend if matrix-host matrix-path

# Frontend: MATRIX_frontend (Listining * Port 8448)
frontend MATRIX_frontend
    bind *:8448 name *:8448 alpn h2,http/1.1 ssl  crt-list /tmp/haproxy/ssl/6256daae2378c2.17892750.certlist
    bind [::]:8448 name [::]:8448 alpn h2,http/1.1 ssl  crt-list /tmp/haproxy/ssl/6256daae2378c2.17892750.certlist
    mode http
    option http-keep-alive
    default_backend MATRIX_backend
    # tuning options
    timeout client 1000s

    # logging options
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
      http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
      http-request set-header X-Forwarded-For %[src]

# Frontend: SSH_frontend (Listining * Port 22)
frontend SSH_frontend
    bind *:22 name *:22 alpn h2,http/1.1
    bind [::]:22 name [::]:22 alpn h2,http/1.1
    mode tcp
    # tuning options
    timeout client 1000s

    # logging options

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    server SSL_server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: WEBSERVER_backend ()
backend WEBSERVER_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    acl restricted_page path_beg /wp-admin
    acl auth_ok http_auth(Allowedusers)
    http-request auth if restricted_page !auth_ok
   
    http-reuse safe
    server WEBSERVER_server 192.168.1.100:80 send-proxy-v2 check-send-proxy
    server WEBSERVER_server_ipv6 XXX:168:a774::2000:80 send-proxy-v2 check-send-proxy

# Backend: NAS_backend ()
backend NAS_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-reuse safe
    server NAS_server 192.168.1.118:80
    server NAS_server_ipv6 XXX:168:a774::1000:80

# Backend: WEBSERVER_SSL_backend ()
backend WEBSERVER_SSL_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
   
    http-reuse safe
    server WEBSERVER_server_ssl 192.168.1.100:443
    server WEBSERVER_server_ssl_ipv6 XXX:168:a774::2000:443

# Backend: MOPIDY_backend ()
backend MOPIDY_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    acl auth_ok http_auth(list_6245eeb66d3ab2.08976803)
    http-request auth if !auth_ok
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    acl is_root path -i /
    redirect code 301 location /iris if is_root
    http-reuse safe
    server MOPIDY_server 192.168.1.100:6680

# Backend: MATRIX_backend ()
backend MATRIX_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
      http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
      http-request set-header X-Forwarded-For %[src]
    http-reuse safe
    server MATRIX_server 192.168.1.100:8008
    server MATRIX_server_ipv6 XXX:168:a774::2000:8008

# Backend: KVM_backend ()
backend KVM_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-reuse safe
    server KVM_server 192.168.1.105:80

# Backend: SYNC_backend ()
backend SYNC_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
      http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
      http-request set-header X-Forwarded-For %[src]
    http-reuse safe
    server SYNC_server 192.168.1.100:5050

# Backend: ROUTER_SSH_backend ()
backend ROUTER_SSH_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    server ROUTER_SSH_Server 192.168.1.1:22
    server ROUTER_SSH_Server_ipv6 XXX:168:a774::1000:22

# Backend: NAS_SSH_backend ()
backend NAS_SSH_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    server NAS_server_ipv6 XXX:168:a774::1000:80
    server NAS_SSH_server 192.168.1.118:22

# Backend: KVM_SSH_backend ()
backend KVM_SSH_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    server KVM_SSH_server 192.168.1.105:22

# Backend: SERVER_SSH_backend ()
backend SERVER_SSH_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 1000s
    timeout server 1000s
    server SERVER_SSH_server 192.168.1.100:22
    server SERVER_SSH_server_ipv6 XXX:168:a774::2000:22


Could someone say why my SSH service points everytime to the Router and doesn't split to ssh, ssh.kvm, ssh.server, ssh.nas?

My map file looks like

Code Select Expand


#public access subdomains
flood WEBSERVER_backend
kvm KVM_backend
nas WEBSERVER_backend
grafana WEBSERVER_backend
phpmyadmin WEBSERVER_backend
speedtestserver WEBERSERVER_backend
cloud NAS_backend
dav NAS_backend
stefan NAS_backend
mopidy MOPIDY_backend
git WEBSERVER_backend
chat MATRIX_backend
admin WEBSERVER_backend
sync SYNC_backend
ssh.nas NAS_SSH_backend
ssh.server SERVER_SSH_backend
ssh ROUTER_SSH_backend
ssh.kvm KVM_SSH_backend


Is a frontend for port 22 necessary?
Thanks for advices.

Now looks like this but doesn't solve the problem

Yes. The aliases has a IPv4 and IPv6. I will check later! Thanks for the Input.

Hi!

I have two rules. Exactly the same for 5060 and 8080 Port





curl works for IPv4 but not for IPv6. The SERVER has a IPv4 (192.168.1.100) and IPv6(2a02:XXX:a774:2000)

A direct curl on SERVER works with IPv4/6 but a curl on ROUTER IPv6 (192.168.1.1/2a02:XXX:a774::1) doesn't work but IPv4 works

Code Select Expand


[morta@lapt0p ~]$ curl -v6 http://[2a02:XXX:a774::1]:5060
*   Trying 2a02:XXX:a774::1:5060...
* connect to 2a02:XXX:a774::1 port 5060 failed: Die Wartezeit für die Verbindung ist abgelaufen
* Failed to connect to 2a02:XXX:a774::1 port 5060 after 129960 ms: Die Wartezeit für die Verbindung ist abgelaufen
* Closing connection 0
curl: (28) Failed to connect to 2a02:XXX:a774::1 port 5060 after 129960 ms: Die Wartezeit für die Verbindung ist abgelaufen
[morta@lapt0p ~]$ curl -v6 http://[2a02:XXX:a774::2000]:5060
*   Trying 2a02:XXX:a774::2000:5060...
* Connected to 2a02:XXX:a774::2000 (2a02:XXX:a774::2000) port 5060 (#0)
> GET / HTTP/1.1
> Host: [2a02:XXX:a774::2000]:5060
> User-Agent: curl/7.85.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sun, 25 Sep 2022 14:31:34 GMT
< Connection: Keep-Alive
< Content-Type: text/html
< Content-Length: 109
<
<html><head><title>OoklaServer</title></head><body><h1>OoklaServer</h1><p>It worked!<br /></p></body></html>
* Connection #0 to host 2a02:XXX:a774::2000 left intact



What I'm doing wrong?

After deleting every rule with port 80 is working fine so far

Code Select Expand

There were error(s) loading the rules: /tmp/rules.debug:88: address family (inet/inet6) undefined - The line in question reads [88]: binat on ixl1 from $SERVER to any -> $SERVER

Line 88 in /tmp/rules.debug is

Code Select Expand

# rdr pass on ixl1 inet proto tcp from {any} to {(ixl1)} port {80} -> $SERVER port 80

How to fix this error?

SERVER is a aliases
ixl1 is the WAN port 

Hi

I got following error in debug and verbose mode

igmpproxy -d -v /usr/local/etc/igmpproxy.conf

Code Select Expand


adding VIF, Ix 0 Fl 0x0 IP 0xeaeac355 ixl1, Threshold: 1, Ratelimit: 0
adding VIF, Ix 1 Fl 0x0 IP 0x0101a8c0 bridge0, Threshold: 1, Ratelimit: 0
Joining group 224.0.0.2 on interface bridge0
Joining group 224.0.0.22 on interface bridge0
sendto to 224.0.0.1 on 192.168.1.1; Errno(13): Permission denied
RECV Membership query   from 192.168.1.1     to 224.0.0.1
RECV V3 member report   from 192.168.1.104   to 224.0.0.22
Inserted route table entry for 239.255.255.250 on VIF #1
Joining group 239.255.255.250 on interface ixl1
RECV V3 member report   from 192.168.1.104   to 224.0.0.22
Updated route entry for 239.255.255.250 on VIF #1
The IGMP message was local multicast. Ignoring.
RECV V3 member report   from 192.168.1.104   to 224.0.0.22
Updated route entry for 239.255.255.250 on VIF #1
RECV V3 member report   from 192.168.1.104   to 224.0.0.22
Updated route entry for 239.255.255.250 on VIF #1
RECV V2 member report   from 192.168.1.1     to 224.0.0.2
The IGMP message was from myself. Ignoring.
RECV V2 member report   from 192.168.1.1     to 224.0.0.22
The IGMP message was from myself. Ignoring.
sendto to 224.0.0.1 on 192.168.1.1; Errno(13): Permission denied
RECV Membership query   from 192.168.1.1     to 224.0.0.1
RECV V2 member report   from 192.168.1.1     to 224.0.0.22
The IGMP message was from myself. Ignoring.

How I can fix this error?

Code Select Expand

sendto to 224.0.0.1 on 192.168.1.1; Errno(13): Permission denied

Hi Morta,

Read the tutorial.

1. As far as IGMP Proxy is concerned, you have two options. Either use 77.109.129.0/25 or the four (4) hosts mentioned ( 77.109.129.16/32, 77.109.129.17/32, 77.109.129.18/32, 77.109.129.19/32 ). In short ... either use the range that also contains the four hosts or use the exact four hosts. You configured both.

2. Does your internal network use 192.168.1.0/24? If not ... alter the LAN_DOWN range to match your internal network range.

3. Did you configure the needed firewall rules? If not, please do.

Good luck!  :)

1. Ok I will delete the range. Thx

2. Yes my network use 192.168.1.0/24

3. I have following Firewall rules as floating for WAN

Code Select Expand


IPv4 IGMP WAN address * 224.0.0.0/4 * * * OUT
IPv4 IGMP WAN address * 224.0.0.0/4 * * * IN
IPv4 UDP WAN address * 224.0.0.0/4 * * * IN


Rules for WAN Firewall

Code Select Expand


IPv4 IGMP WAN net * 224.0.0.0/4 * * * Allow IGMP Multicast Traffic
IPv4 PIM WAN net * 224.0.0.0/4 * * * Allow PIM Traffic
IPv4 UDP 77.109.129.16 * 239.0.0.0/8 5000 * * Allow init7 Multicast Traffic
IPv4 UDP 77.109.129.17 * 239.0.0.0/8 5000 * * Allow init7 Multicast Traffic
IPv4 UDP 77.109.129.18 * 239.0.0.0/8 5000 * * Allow init7 Multicast Traffic
IPv4 UDP 77.109.129.19 * 239.0.0.0/8 5000 * * Allow init7 Multicast Traffic



And for LAN

Code Select Expand


Automatically generated rules
IPv4 * LAN net * * * * * Default allow LAN to any rule
IPv4 IGMP LAN net * * * * * Default allow LAN to any rule
IPv6 * LAN net * * * * * Default allow LAN IPv6 to any rule
IPv4+6 * * * * * * * Default outbound rule



Have I to add more or do you see any errors?

PS: Multicast NOR Unicast stream are working out of box. So can't be my settings?!

Sorry my english is not the best

I did this tutorial

https://forum.opnsense.org/index.php?topic=17865.0

Are you see any different to your tutorial?

So the outage could be also from the ISP.
They have a lot of problem with the multicast stream.

I get a stream but the stream is stocking and pixelated.

So what could be the reason?

Wrong tutorial?
ISP Problem?
Realtek card?

Do you understand now what I try to say?

I search other user with TV7 and his experience!