Messages

Hi,

I searched for similar posts, but could not find an example use case, so posting here.

I have opnsense with the following

- vtnet0 - ISP1 
- vtnet1 - LAN 
- vtnet2 - ISP2 

ISP1 is the default gateway, and all machines in the LAN  use send/receive using ISP1.

suppose the IPs are like this.

ISP1 is 1.1.1.1
ISP2 is 2.2.2.2
LAN  is 192.168.0.1
test-machine is 3.3.3.3 ( outside ip)

I am using cilium bgp in k8s and announcing the load balancer ip to opnsense.
I can see the 10.101.101.0/32 - announced OK.


since 10.101.101.0/32 is in route in opnsense, if I do http/curl 10.101.101.0, i get nginx test page from opnsense and all machines in the lan.

i also have following forwarding done in opnsense.

1.1.1.1 80/443 ->  10.101.101.0 80/443
2.2.2.2 80/443 ->  10.101.101.0 80/443

from 3.3.3.3 if I do curl 1.1.1.1, i get the nginx OK.
from 3.3.3.3 if I do curl 2.2.2.2, i do not get the page.


what i see when I try to do curl 2.2.2.2 is

vtnet2(ISP2) -> 3.3.3.3.xxxx -> 2.2.2.2.80
vtnet1(LAN) -> 10.101.101.0.80 ->  3.3.3.3.xxxx
vtnet0(ISP1) -> 2.2.2.2.80  ->  3.3.3.3.xxxx

here 2.2.2.2 ( ip from isp2) is seen as trying to go out via isp1

how do I add route/policy such that when  request is made to 2.2.2.2(ISP2), it is returned via ISP2 and not via ISP1.



Thanks

Hi,

opnsense has ipv6 ip address.
ping to this IP works fine.

nginx is listening on ipv6 .. .so curl "https://[::1]:8443" returns the website just fine

from the firewall, i have      v6:443 ->  ::1:8443 forwarded.  This is exactly the same as v4 forward which works fine.

When I do https://v6:443/ , from tcpdump I see that the packets are reaching the firewall and the port ,but it does not get forwarded to ::1:8443 and there is nothing more in tcdump or nginx logs or in firewall logs.  Same for port 80 forwarding.

Looks like the  requests comes to (WAN) v6:443 and disappears from there.

The behaviour is the same with firewall disabled "pfctl -d "

Can someone provide me pointers on how to fix/troubleshoot this ?


Many thanks

Thanks,
Will try.

The controller is PERC H730P Mini

Hi,

we are not able to install this version at all.

This is on a dell server R730  with raid card ( which we are not  able to bypass)

There are 3 possible configs from the bios for the raid card and disks.

1. raid controller on raid mode and  disks on raid
2. raid controller on raid mode and disks non-raid
3. raid controller on hba mode and disks non-raid

on the opnsense side, the config options are:

zfs (stripe) on  top of raid
zfs (mirror ) on non raid
zfs (mirror ) on hba , non-raid
ufs install on first disk

All of these 4 options fail.

Research points to using mrsas and disabling mfi driver, but I was not able to figure out how to do it from the installation iso. I was hoping to tackle this after installation, but it does not install at all and just hangs.

Screenshots
- zfs mirror takes ages with disks on non-raid mode .. in 6 hours, it moved 1%
- rest of the 4 modes give io errors as in the screenshot

If somene else has dell servers and the raid card, can you please let me know what raid card you are using, or what is done to install it. In 24.1 the only possible way to install was doing non-raid mode and zfs mirror, but its full of erros and the system is slow with constant errors as in the screenshot.

also what would be the reason to spawn dozens of /usr/local/sbin/pluginctl  .. can't it be done every x minute and data saved to some file which the next login picks.

also have found that /usr/local/opnsense/scripts/nginx/ngx_autoblock.php  starts another process if the first one is still running .. so sometimes there are 3-4 /usr/local/opnsense/scripts/nginx/ngx_autoblock.php processes that have not finished processing.

Hi,

reboot has no effect.. when it comes up the first time, during ssh this is shown ..

ssh
Last login: Mon Jul 22 18:32:55 2024 from x.x.x.x
----------------------------------------------
|      Hello, this is OPNsense 24.1          |         @@@@@@@@@@@@@@@
|                                            |        @@@@         @@@@
| Website:   https://opnsense.org/        |         @@@\\\   ///@@@
| Handbook:   https://docs.opnsense.org/   |       ))))))))   ((((((((
| Forums:   https://forum.opnsense.org/  |         @@@///   \\\@@@
| Code:      https://github.com/opnsense  |        @@@@         @@@@
| Twitter:   https://twitter.com/opnsense |         @@@@@@@@@@@@@@@


then it takes at least 5 minutes minutes to finally login and show the menu options ..
ps aux after reboot just has the default nginx proces of php-fpm for www and webgui

when i load the browser, it shows the login screen, but as soon as i press the login , top/ps acts like posted above ..  dozens of processes are spawned

Hi,

Our server is crawling. GUI login takes ages and cli also takes ages to login.
I have disabled everything from the GUI except system info and services.

server is bare metal  ..  20 cores @3.1ghz  40threads
there is no disk errors.


top

last pid: 59277;  load averages: 45.60, 37.17, 24.50                                      up 0+02:15:08  20:09:29
131 processes: 47 running, 84 sleeping
CPU: 98.8% user,  0.0% nice,  1.2% system,  0.0% interrupt,  0.0% idle
Mem: 8950M Active, 3803M Inact, 1779M Wired, 40K Buf, 172G Free
ARC: 413M Total, 124M MFU, 248M MRU, 171K Anon, 2194K Header, 39M Other
     149M Compressed, 325M Uncompressed, 2.18:1 Ratio
Swap: 16G Total, 16G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
87555 root          1 102    0   250M   198M CPU21   21   3:13 100.03% php
86056 root          1  98    0   250M   197M CPU35   35   8:01  99.72% php
37729 root          1 102    0   248M   197M CPU14   14   5:13  99.67% php
89499 root          1 100    0   234M   181M CPU9     9   2:40  99.60% php
7784 root          1 102    0   226M   174M CPU16   16   1:54  99.59% php
2216 root          1 101    0   228M   176M CPU13   13   2:07  99.59% php
  463 root          1 101    0   238M   187M CPU25   25   2:14  99.59% php
26232 root          1 103    0   228M   176M CPU38   38   1:25  99.59% php
90693 root          1 101    0   230M   177M CPU18   18   2:12  99.59% php
27078 root          1 100    0   244M   193M CPU34   34   5:46  99.59% php
39785 root          1 101    0   240M   189M CPU4     4   5:03  99.58% php
78831 root          1 102    0   462M   377M CPU31   31   5:34  99.57% php-cgi
2151 root          1 101    0   252M   200M CPU23   23   7:34  99.57% php
41256 root          1 102    0   214M   161M CPU11   11   0:43  99.57% php
45343 root          1  91    0   236M   183M CPU20   20   4:53  99.57% php
77558 root          1 101    0   242M   189M CPU3     3   3:16  99.57% php
26374 root          1 102    0   238M   187M CPU33   33   6:07  99.56% php
23089 root          1 101    0   220M   169M CPU7     7   1:20  99.56% php
43177 root          1 100    0   246M   194M CPU2     2   5:12  99.56% php
60214 root          1 100    0   244M   192M CPU17   17   4:09  99.56% php
99979 root          1 102    0   228M   176M RUN     10   2:08  99.56% php
91377 root          1 100    0   238M   185M CPU26   26   2:05  99.55% php
88932 root          1 102    0   248M   195M CPU28   28   2:58  99.55% php
3427 root          1  99    0   244M   192M CPU37   37   7:25  99.52% php
40085 root          1  97    0   218M   167M CPU32   32   0:46  99.30% php
32528 root          1 101    0   248M   196M CPU6     6   5:22  99.30% php
77016 root          1 101    0   240M   187M CPU0     0   3:27  99.29% php
78055 root          1  97    0   236M   184M CPU24   24   3:09  87.85% php
40818 root          1  95    0   234M   182M RUN     30   5:02  79.92% php
43349 root          1  91    0   206M   154M CPU12   12   0:22  77.87% php
35170 root          1  95    0   218M   165M CPU29   29   0:44  77.26% php
41873 root          1 100    0   216M   164M CPU39   39   0:42  76.67% php
42431 root          1  97    0   214M   161M RUN     36   0:32  74.81% php
32896 root          1  94    0   240M   189M CPU8     8   5:11  74.25% php
69261 root          1  98    0   244M   193M RUN      5   3:56  73.62% php
76982 root          1 101    0   258M   205M RUN     27   3:32  73.00% php
79037 root          1  93    0   236M   185M CPU15   15   3:10  71.82% php
2075 root          1 101    0   260M   206M RUN      1   7:37  71.81% php
77777 root          1 100    0   311M   264M CPU19   19  20:52  70.88% php-cgi
88751 root          1  97    0   248M   196M CPU22   22   2:53  70.60% php
56147 root          1  89    0   204M   152M RUN     19   0:17  56.47% php
38002 root          1  93    0   216M   164M CPU5     5   0:53  51.29% php
55126 root          1  92    0   250M   199M CPU1     1   4:46  49.50% php
37552 root          1  94    0   246M   195M CPU27   27   5:01  48.89% php
70493 root          1  94    0   232M   180M CPU30   30   4:00  48.41% php
3551 root          1  94    0   244M   192M CPU36   36   7:03  47.68% php
  288 root         43  20    0   627M   353M accept  29   2:39   1.05% python3.11
59277 root          1  20    0    14M  3900K CPU10   10   0:00   0.22% top
35395 root          4  20    0   938M    19M select  21   0:02   0.13% bgpd

# ps ax | grep php
1122  -  Ss      0:00.19 php-fpm: master process (/usr/local/etc/php-fpm.conf) (php-fpm)
1620  -  I       0:00.00 php-fpm: pool webgui (php-fpm)
1748  -  I       0:00.00 php-fpm: pool webgui (php-fpm)
1817  -  I       0:00.00 php-fpm: pool www (php-fpm)
2166  -  I       0:00.00 php-fpm: pool www (php-fpm)
2216  -  R       4:42.36 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-openvpn 157@lagg0_vlan224 BACKUP
3427  -  R      10:04.91 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/50-frr 135@lagg0_vlan224 BACKUP
3551  -  R       9:34.47 /usr/local/bin/php /usr/local/sbin/pluginctl -S
7784  -  R       4:25.77 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
10447  -  Rs      1:14.12 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
20783  -  R       1:15.91 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-openvpn 241@lagg0_vlan224 BACKUP
21885  -  R       1:03.06 /usr/local/bin/php /usr/local/opnsense/scripts/Wireguard/wg-service-control.php -a conf
22572  -  R       0:59.95 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/50-frr 137@lagg0_vlan224 BACKUP
23038  -  R       0:54.80 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
23089  -  Rs      3:56.91 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
23495  -  R       1:00.05 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-ppp 151@lagg0_vlan224 BACKUP
24073  -  R       1:00.77 /usr/local/bin/php /usr/local/sbin/pluginctl -c crl
24759  -  R       0:42.07 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
26219  -  R       0:37.45 /usr/local/bin/php /usr/local/sbin/pluginctl -c crl
26232  -  R       4:04.92 /usr/local/bin/php /usr/local/sbin/pluginctl -S
26374  -  R       8:30.44 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-openvpn 153@lagg0_vlan224 BACKUP
26837  -  R       0:35.64 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-ppp 155@lagg0_vlan224 BACKUP
27078  -  R       8:16.72 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
28152  -  R       0:32.77 /usr/local/bin/php /usr/local/opnsense/scripts/Wireguard/wg-service-control.php -a conf
28583  -  R       0:26.58 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/50-frr 139@lagg0_vlan224 BACKUP
32140  -  Rs      0:27.05 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
32896  -  R       7:48.95 /usr/local/bin/php /usr/local/sbin/pluginctl -c crl
35170  -  R       3:04.46 /usr/local/bin/php /usr/local/sbin/pluginctl -S
37552  -  S       7:14.67 /usr/local/bin/php /usr/local/sbin/pluginctl -S
37729  -  S       6:49.08 /usr/local/bin/php /usr/local/sbin/pluginctl -S
38002  -  R       3:30.64 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
39785  -  Rs      7:43.50 /usr/local/bin/php /usr/local/etc/rc.expireaccounts
40085  -  R       3:21.19 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
40818  -  Rs      7:35.82 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
41256  -  R       3:13.19 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
41873  -  R       3:10.05 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
42431  -  R       3:01.50 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
43177  -  S       7:01.00 /usr/local/bin/php /usr/local/sbin/pluginctl -S
43349  -  Rs      2:50.71 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
43511  -  R       0:18.16 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
45343  -  R       7:23.18 /usr/local/bin/php /usr/local/sbin/pluginctl -S
55126  -  S       5:42.35 /usr/local/bin/php /usr/local/sbin/pluginctl -S
56147  -  R       2:49.63 /usr/local/bin/php /usr/local/etc/rc.syshook.d/carp/20-openvpn 159@lagg0_vlan224 BACKUP
70493  -  R       6:31.91 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
75925  -  Is      0:00.02 /usr/local/bin/php-cgi
76048  -  R       2:10.76 /usr/local/bin/php /usr/local/sbin/pluginctl -S
76263  -  Is      0:00.02 /usr/local/bin/php-cgi
76763  -  Rs      2:02.20 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
76781  -  R      15:26.64 /usr/local/bin/php-cgi
77016  -  R       5:55.53 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
77052  -  S      17:01.47 /usr/local/bin/php-cgi
77558  -  R       5:57.04 /usr/local/bin/php /usr/local/sbin/pluginctl -S
77586  -  I       7:07.25 /usr/local/bin/php-cgi
77777  -  R      21:30.97 /usr/local/bin/php-cgi
78055  -  R       5:45.46 /usr/local/bin/php /usr/local/sbin/pluginctl -S
78274  -  I       2:53.81 /usr/local/bin/php-cgi
78831  -  R       8:06.96 /usr/local/bin/php-cgi
78999  -  R       2:04.85 /usr/local/bin/php /usr/local/sbin/pluginctl -S
79037  -  Rs      5:47.07 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
85633  -  R       1:35.71 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
86117  -  R       1:43.09 /usr/local/bin/php /usr/local/sbin/pluginctl -S
88751  -  S       3:57.07 /usr/local/bin/php /usr/local/sbin/pluginctl -S
88932  -  S       4:02.84 /usr/local/bin/php /usr/local/sbin/pluginctl -S
89499  -  R       5:09.13 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
90692  -  R       1:36.14 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
90693  -  Rs      4:48.34 /usr/local/bin/php /usr/local/opnsense/scripts/nginx/ngx_autoblock.php
90973  -  R       1:29.91 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
91216  -  R       1:26.67 /usr/local/bin/php /usr/local/opnsense/scripts/ipsec/get_legacy_vti.php
99979  -  R       4:37.45 /usr/local/bin/php /usr/local/sbin/pluginctl -c crl

how to find out the root cause of whats causing this.


Thanks

Hi,

In my nginx error logs I have something like this

2024/06/20 22:05:37 [error] 75685#101233: *77740 upstream timed out (60: Operation timed out) while reading response header from upstream, client: A.B.C.D  server: www.example.com, request:  "GET / "..

I could not find any settings related to this in the GUI. On the config file, below location for the specific domain,  i have under the proxy_ settings  include 4f3bf9ec-0322-4045-b9bc-4b26f43e9ab7_post/*.conf entry.
I created the folder, created domain.conf file and put

proxy_connect_timeout 300;


but I still get the 60s error above.

What is the recommended way to set  proxy_connect_timeout  ?

Thanks

Hi All,

How do I set and forward $ssl_client_fingerprint via nginx to the backend ?

on the _post/site.conf , i tried

proxy_set_header X-SSL-Client-Fingerprint $ssl_client_fingerprint; 
Also tried  $http_ssl_ja3_hash  and $http_ssl_ja3 but none seem to be set.

Also set ssl-client-verify to be on , but it does not set the header.

Thanks

figured it out :)

Hi,

I have location / and one website with multi domain SSL.

the SSL are for  domain.nl domain.fr domain.de domain.eu  etc
what I want to do is something like this.

if url is http( or https)  :// domain.(nl| fr|de| eu), redirect it to https://www.domain.com/$1/$2 ,  where $1 = nl fr de eu etc and $2 is reset of the query string ..
so if url comes to http://domain.nl/abc/123 , it will redirect to https://www.domain.com/nl/abc/123

This is the regex I am using

Code Select Expand


^(?:(?:http:\/\/|https:\/\/)?(?:www\.)?)?domain\.(nl|fr|de|eu)(\/.*)?$
and the redirect is
https://www.domain.com/$1/$2   

However, this does not work .. I have tried the regex in both location block as well as http server block .

Can someone please help ?


Thanks