Hi All,
Updated opnsense yesterday on my Protectli Vault FW4C and have had all sorts of issues since.
Upgrade from 26.1... Left me with no internet access (although opnsense could ping etc so had internet) I asssume it was dns related and after failing to resolve that after an hour or two I just did a fresh clean install of 26.7 as my setup is pretty straight forward.
Only issue is I seems to have some sort of memory leak. I can watch memory uisage keep climbing until it reaches near 100% and its gets unstable until I do a reboot.
Looking at activity under system - dignostics i can see multiple root processes that appear to be python steadily use more memory:
/usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.13){python3.13}
I also seems to end up with hundreds of these entries, all appearing to use more memory as time progresses?
I'm far from an expert on this stuff so not sure what this is or how to resolve it.
Any suggestions much appreciated!
Thanks
What does this return, it should show what calls configd:
# ps -faxdv
Hi, thanks for the reply.
Output below:
root@OPNsense:~ # ps -faxdv
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
0 DLs 0:04.01 127 127 0 0 2400 - 0 0.0 0.0 [kernel]
1 SLs 0:00.11 0 127 12 12392 1180 - 716 0.0 0.0 - /sbin/ini
51569 Ss 0:01.36 1 127 0 97872 53328 - 1104 0.0 0.6 |-- /usr/lo
82414 Ss 0:00.00 0 0 0 21740 8888 - 500 0.0 0.1 |-- /usr/lo
37962 S 0:00.10 1 127 0 18148 5012 - 336 0.0 0.1 |-- /usr/lo
53502 Ss 0:00.02 13 127 0 22120 9504 - 284 0.0 0.1 |-- sshd: /
78274 Ss 0:00.02 13 26 0 22676 10096 - 684 0.0 0.1 | `-- sshd-
29313 S 0:00.01 0 13 0 22676 10384 - 684 0.0 0.1 | `-- ssh
29634 Ss 0:00.01 11 13 0 14672 3104 - 124 0.0 0.0 | `-- /
52348 S 0:00.02 0 11 0 15024 4144 - 336 0.0 0.1 | `--
88541 R+ 0:00.00 127 0 0 14744 3420 - 24 0.0 0.0 | `
64763 S 0:00.62 0 127 1 26036 12780 - 224 0.0 0.2 |-- /usr/lo
28461 Is 0:00.09 127 127 0 199960 39276 - 6460 0.0 0.5 | |-- /usr/
28991 I 0:00.20 127 127 0 200088 43464 - 6460 0.0 0.5 | | |-- /us
29262 I 0:00.33 127 127 2 208360 52544 - 6460 0.0 0.6 | | |-- /us
29328 I 0:00.43 127 127 2 208360 52948 - 6460 0.0 0.6 | | |-- /us
29644 I 0:00.00 127 127 0 199960 39272 - 6460 0.0 0.5 | | |-- /us
29822 I 0:00.00 127 127 0 199960 39272 - 6460 0.0 0.5 | | `-- /us
65621 Is 0:00.10 127 127 32 199960 36012 - 6460 0.0 0.4 | |-- /usr/
67505 I 0:00.07 127 127 0 200024 38760 - 6460 0.0 0.5 | | |-- /us
67836 I 0:00.00 127 127 0 199960 36016 - 6460 0.0 0.4 | | |-- /us
68663 I 0:00.09 127 127 0 200024 39828 - 6460 0.0 0.5 | | |-- /us
69968 I 0:00.00 127 127 0 199960 36016 - 6460 0.0 0.4 | | |-- /us
70634 I 0:00.00 127 127 0 199960 36016 - 6460 0.0 0.4 | | `-- /us
66813 Is 0:00.09 127 127 8 199960 35696 - 6460 0.0 0.4 | |-- /usr/
72578 I 0:00.10 127 127 0 200088 39452 - 6460 0.0 0.5 | | |-- /us
72617 I 0:00.00 127 127 0 199960 35704 - 6460 0.0 0.4 | | |-- /us
72775 I 0:00.00 127 127 0 199960 35704 - 6460 0.0 0.4 | | |-- /us
72826 I 0:00.12 127 127 0 200024 39896 - 6460 0.0 0.5 | | |-- /us
73619 I 0:00.00 127 127 0 199960 35704 - 6460 0.0 0.4 | | `-- /us
66918 Is 0:00.09 127 127 18 199960 35788 - 6460 0.0 0.4 | `-- /usr/
68586 I 0:00.52 127 127 36 200168 43852 - 6460 0.0 0.5 | |-- /us
69125 I 0:00.18 127 127 0 212636 41832 - 6460 0.0 0.5 | |-- /us
70584 I 0:00.00 127 127 0 199960 35792 - 6460 0.0 0.4 | |-- /us
70859 I 0:00.00 127 127 0 199960 35792 - 6460 0.0 0.4 | |-- /us
71621 I 0:00.00 127 127 0 199960 35792 - 6460 0.0 0.4 | `-- /us
38098 Is 0:00.00 92 127 0 14432 2884 - 64 0.0 0.0 |-- dhclien
70038 Ss 0:00.22 0 127 0 14140 2696 - 92 0.0 0.0 |-- /usr/lo
83211 S 0:00.00 0 0 0 14672 2960 - 124 0.0 0.0 | `-- /bin/
88508 R 0:00.01 127 0 0 22232 9760 - 4 0.0 0.1 | `-- /us
957 Ss 0:00.02 13 127 0 15700 4452 - 36 0.0 0.1 |-- /sbin/d
38890 SCs 0:00.05 1 127 0 14280 2760 - 28 0.0 0.0 |-- /usr/sb
41504 Is 0:00.00 127 127 0 14280 2728 - 28 0.0 0.0 |-- rtsold:
42535 Is 0:00.00 127 127 0 14280 2736 - 28 0.0 0.0 |-- rtsold:
43073 Is 0:00.00 127 127 0 14280 2728 - 28 0.0 0.0 |-- rtsold:
43915 Ss 0:00.03 1 127 0 14280 2820 - 28 0.0 0.0 |-- rtsold:
83565 Is 0:00.02 38 127 0 14208 2756 - 28 0.0 0.0 |-- /usr/sb
3847 Is+ 0:00.00 127 127 0 14132 2548 - 20 0.0 0.0 |-- /usr/li
3914 Is+ 0:00.00 127 127 0 14132 2544 - 20 0.0 0.0 |-- /usr/li
4063 Is+ 0:00.00 127 127 0 14132 2556 - 20 0.0 0.0 |-- /usr/li
4068 Is+ 0:00.00 127 127 0 14132 2552 - 20 0.0 0.0 |-- /usr/li
4256 Is+ 0:00.00 127 127 0 14132 2556 - 20 0.0 0.0 |-- /usr/li
4355 Is+ 0:00.00 127 127 0 14132 2552 - 20 0.0 0.0 |-- /usr/li
4506 Is+ 0:00.00 127 127 0 14132 2548 - 20 0.0 0.0 |-- /usr/li
4845 Is+ 0:00.00 127 127 0 14132 2556 - 20 0.0 0.0 |-- /usr/li
80717 Ss 0:00.00 0 0 0 14596 3056 - 8 0.0 0.0 |-- /usr/lo
34775 Is 0:00.00 92 127 0 14112 2492 - 8 0.0 0.0 |-- daemon:
36073 SC 0:00.03 1 127 2 14436 2996 - 64 0.0 0.0 | `-- dhcli
61417 Is 0:00.01 92 127 0 14432 2956 - 64 0.0 0.0 | `-- dhc
53905 I 0:00.00 127 127 0 26796 11560 - 4 0.0 0.1 |-- /usr/lo
54660 Ss 0:03.34 0 127 14 48044 16572 - 4 0.2 0.2 | `-- /usr/
6281 S 0:00.36 0 127 0 28844 15528 - 4 0.0 0.2 | |-- /us
64177 S 0:00.93 0 127 0 28856 15688 - 4 0.0 0.2 | `-- /us
441 Is 0:01.93 127 127 0 29588 16352 - 4 0.0 0.2 `-- /usr/lo
444 S 0:20.49 0 127 71 162736 81664 - 4 0.2 1.0 `-- /usr/
3349 I 0:00.11 86 86 0 70932 39660 - 6404 0.0 0.5 |-- /us
5750 I 0:00.11 103 103 0 70932 39676 - 6404 0.0 0.5 |-- /us
5760 I 0:00.11 94 94 0 70932 39656 - 6404 0.0 0.5 |-- /us
6544 I 0:00.11 76 77 0 70932 39640 - 6404 0.0 0.5 |-- /us
7069 I 0:00.12 127 127 0 70932 39660 - 6404 0.0 0.5 |-- /us
7239 I 0:00.11 67 67 0 70932 39640 - 6404 0.0 0.5 |-- /us
7817 I 0:00.11 127 127 0 70932 39644 - 6404 0.0 0.5 |-- /us
10770 I 0:00.12 42 42 0 70932 39640 - 6404 0.0 0.5 |-- /us
11171 I 0:00.11 34 34 0 70932 39672 - 6404 0.0 0.5 |-- /us
11835 I 0:00.12 49 49 0 70932 39672 - 6404 0.0 0.5 |-- /us
12459 I 0:00.11 66 66 0 70932 39668 - 6404 0.0 0.5 |-- /us
12618 I 0:00.11 93 93 0 70932 39672 - 6404 0.0 0.5 |-- /us
12928 S 0:00.11 6 6 0 70932 39696 - 6404 0.4 0.5 |-- /us
13286 I 0:00.12 74 75 0 70932 39668 - 6404 0.0 0.5 |-- /us
13791 I 0:00.12 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
14099 I 0:00.11 85 85 0 70932 39656 - 6404 0.0 0.5 |-- /us
14840 I 0:00.12 119 119 0 70932 39672 - 6404 0.0 0.5 |-- /us
15244 I 0:00.11 24 25 0 70932 39640 - 6404 0.0 0.5 |-- /us
18315 I 0:00.11 127 127 0 70932 39664 - 6404 0.0 0.5 |-- /us
18416 I 0:00.11 127 127 0 70932 39660 - 6404 0.0 0.5 |-- /us
18491 I 0:00.11 57 57 0 70932 39684 - 6404 0.0 0.5 |-- /us
19694 I 0:00.13 41 41 0 70932 39680 - 6404 0.0 0.5 |-- /us
19965 I 0:00.12 127 127 0 70932 39656 - 6404 0.0 0.5 |-- /us
20455 I 0:00.11 83 83 0 70932 39656 - 6404 0.0 0.5 |-- /us
20600 I 0:00.11 109 109 0 70932 39684 - 6404 0.0 0.5 |-- /us
20829 S 0:00.12 14 14 0 70932 39660 - 6404 0.1 0.5 |-- /us
21543 I 0:00.10 117 117 0 70932 39676 - 6404 0.0 0.5 |-- /us
21876 I 0:00.12 33 33 0 70932 39672 - 6404 0.0 0.5 |-- /us
22289 I 0:00.12 101 101 0 70932 39652 - 6404 0.0 0.5 |-- /us
22320 I 0:00.11 47 47 0 70932 39676 - 6404 0.0 0.5 |-- /us
22399 I 0:00.11 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
23405 I 0:00.11 64 64 0 70932 39664 - 6404 0.0 0.5 |-- /us
23508 I 0:00.11 73 73 0 70932 39652 - 6404 0.0 0.5 |-- /us
24954 I 0:00.10 127 127 0 70932 39664 - 6404 0.0 0.5 |-- /us
25626 I 0:00.13 55 55 0 70932 39640 - 6404 0.0 0.5 |-- /us
26968 I 0:00.12 23 23 0 70932 39652 - 6404 0.0 0.5 |-- /us
27325 I 0:00.13 108 108 0 70932 39672 - 6404 0.0 0.5 |-- /us
27409 I 0:00.11 31 31 0 70932 39656 - 6404 0.0 0.5 |-- /us
28738 I 0:00.12 127 127 0 70932 39652 - 6404 0.0 0.5 |-- /us
28836 S 0:00.12 13 13 0 70932 39676 - 6404 0.2 0.5 |-- /us
29251 I 0:00.11 46 46 0 70932 39656 - 6404 0.0 0.5 |-- /us
29502 I 0:00.12 127 127 0 70932 39676 - 6404 0.0 0.5 |-- /us
31853 I 0:00.11 21 21 0 70932 39660 - 6404 0.0 0.5 |-- /us
35059 I 0:00.11 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
36455 I 0:00.11 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
37089 S 0:00.11 4 4 0 70932 39664 - 6404 0.5 0.5 |-- /us
39815 I 0:00.11 127 127 0 70932 39668 - 6404 0.0 0.5 |-- /us
40117 S 0:00.11 11 11 0 70932 39664 - 6404 0.2 0.5 |-- /us
42513 I 0:00.11 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
43348 S 0:00.12 3 3 0 70932 39664 - 6404 0.9 0.5 |-- /us
44051 I 0:00.11 44 44 0 70932 39664 - 6404 0.0 0.5 |-- /us
46882 I 0:00.12 39 39 0 70932 39684 - 6404 0.0 0.5 |-- /us
47669 I 0:00.10 127 127 0 70932 39656 - 6404 0.0 0.5 |-- /us
52393 I 0:00.12 92 92 0 70932 39672 - 6404 0.0 0.5 |-- /us
54875 I 0:00.11 72 72 0 70932 39684 - 6404 0.0 0.5 |-- /us
55017 I 0:00.11 99 99 0 70932 39664 - 6404 0.0 0.5 |-- /us
55989 I 0:00.12 81 81 0 70932 39676 - 6404 0.0 0.5 |-- /us
56402 I 0:00.11 125 125 0 70932 39680 - 6404 0.0 0.5 |-- /us
56618 I 0:00.11 127 127 0 70932 39652 - 6404 0.0 0.5 |-- /us
58049 I 0:00.11 62 62 0 70932 39664 - 6404 0.0 0.5 |-- /us
58502 I 0:00.11 53 54 0 70932 39660 - 6404 0.0 0.5 |-- /us
58613 I 0:00.11 115 115 0 70932 39668 - 6404 0.0 0.5 |-- /us
59087 I 0:00.11 127 127 0 70932 39688 - 6404 0.0 0.5 |-- /us
61794 I 0:00.12 80 80 0 70932 39644 - 6404 0.0 0.5 |-- /us
63314 I 0:00.12 70 70 0 70932 39668 - 6404 0.0 0.5 |-- /us
63450 I 0:00.11 90 90 0 70932 39676 - 6404 0.0 0.5 |-- /us
64805 I 0:00.10 98 98 0 70932 39684 - 6404 0.0 0.5 |-- /us
65402 I 0:00.11 127 127 0 70932 39660 - 6404 0.0 0.5 |-- /us
65428 I 0:00.11 127 127 0 70932 39656 - 6404 0.0 0.5 |-- /us
65731 I 0:00.12 123 123 0 70932 39652 - 6404 0.0 0.5 |-- /us
66186 I 0:00.11 29 29 0 70932 39664 - 6404 0.0 0.5 |-- /us
66762 I 0:00.11 127 127 0 70932 39664 - 6404 0.0 0.5 |-- /us
67057 I 0:00.12 61 61 0 70932 39660 - 6404 0.0 0.5 |-- /us
67617 I 0:00.11 78 78 0 70932 39640 - 6404 0.0 0.5 |-- /us
68425 S 0:00.12 19 19 0 70932 39676 - 6404 0.0 0.5 |-- /us
70124 I 0:00.11 127 127 0 70932 39652 - 6404 0.0 0.5 |-- /us
70756 I 0:00.12 127 127 0 70932 39680 - 6404 0.0 0.5 |-- /us
71364 I 0:00.11 37 37 0 70932 39648 - 6404 0.0 0.5 |-- /us
71486 I 0:00.10 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
71728 I 0:00.12 52 52 0 70932 39680 - 6404 0.0 0.5 |-- /us
72209 I 0:00.10 127 127 0 70932 39672 - 6404 0.0 0.5 |-- /us
73306 I 0:00.11 69 69 0 70932 39652 - 6404 0.0 0.5 |-- /us
73898 I 0:00.11 60 60 0 70932 39664 - 6404 0.0 0.5 |-- /us
74176 I 0:00.11 113 113 0 70932 39660 - 6404 0.0 0.5 |-- /us
74521 I 0:00.11 88 88 0 70932 39660 - 6404 0.0 0.5 |-- /us
75461 I 0:00.11 28 28 0 70932 39672 - 6404 0.0 0.5 |-- /us
75851 I 0:00.10 127 127 0 70932 39668 - 6404 0.0 0.5 |-- /us
76233 I 0:00.11 106 106 0 70932 39676 - 6404 0.0 0.5 |-- /us
76296 S 0:00.12 18 18 0 70932 39664 - 6404 0.0 0.5 |-- /us
76674 S 0:00.11 10 10 0 70932 39668 - 6404 0.3 0.5 |-- /us
77007 I 0:00.11 36 36 0 70932 39648 - 6404 0.0 0.5 |-- /us
77503 S 0:00.11 1 1 0 70932 39692 - 6404 1.0 0.5 |-- /us
77749 I 0:00.11 50 50 0 70932 39680 - 6404 0.0 0.5 |-- /us
82823 S 0:00.11 16 16 0 70932 39680 - 6404 0.1 0.5 |-- /us
83208 I 0:00.11 26 26 0 70932 39656 - 6404 0.0 0.5 |-- /us
84561 I 0:00.11 111 112 0 70932 39668 - 6404 0.0 0.5 |-- /us
84617 I 0:00.11 120 121 0 70932 39680 - 6404 0.0 0.5 |-- /us
85324 S 0:00.12 8 9 0 70932 39660 - 6404 0.3 0.5 |-- /us
85856 I 0:00.11 127 127 0 70932 39644 - 6404 0.0 0.5 |-- /us
91424 I 0:00.11 127 127 0 70932 39688 - 6404 0.0 0.5 |-- /us
91870 I 0:00.10 127 127 0 70932 39668 - 6404 0.0 0.5 |-- /us
92653 S 0:00.11 7 7 0 70932 39672 - 6404 0.3 0.5 |-- /us
93813 S 0:00.12 0 127 0 70996 39700 - 6404 0.0 0.5 |-- /us
97136 I 0:00.11 58 58 0 70932 39664 - 6404 0.0 0.5 |-- /us
98575 I 0:00.11 96 96 0 70932 39652 - 6404 0.0 0.5 |-- /us
98859 I 0:00.11 104 105 0 79124 39672 - 6404 0.0 0.5 |-- /us
80894 S 0:00.01 0 0 0 14372 2644 - 24 0.0 0.0 `-- /us
81848 R 0:00.69 127 0 0 59892 41780 - 4 0.0 0.5 `-- /
2 WL 0:00.33 127 127 0 0 64 - 0 0.0 0.0 - [clock]
3 DL 0:00.00 127 127 0 0 80 - 0 0.0 0.0 - [crypto]
4 DL 0:00.00 127 127 0 0 48 - 0 0.0 0.0 - [cam]
5 DL 0:00.00 127 127 0 0 16 - 0 0.0 0.0 - [busdma]
6 DL 0:00.42 0 127 0 0 880 - 0 0.0 0.0 - [zfskern]
7 DL 0:00.24 0 127 0 0 16 - 0 0.0 0.0 - [pf purge
8 DL 0:00.09 0 127 0 0 16 - 0 0.0 0.0 - [rand_har
9 DL 0:00.18 0 127 0 0 48 - 0 0.0 0.0 - [pagedaem
10 DL 0:00.00 127 127 0 0 16 - 0 0.0 0.0 - [audit]
11 RNL 50:44.82 127 127 0 0 64 - 0 299.5 0.0 - [idle]
12 WL 0:00.60 127 127 0 0 176 - 0 0.0 0.0 - [intr]
13 DL 0:00.00 127 127 0 0 48 - 0 0.0 0.0 - [geom]
14 DL 0:00.01 127 127 0 0 80 - 0 0.0 0.0 - [usb]
15 DL 0:00.00 127 127 0 0 16 - 0 0.0 0.0 - [vmdaemon
16 DL 0:00.02 0 127 0 0 80 - 0 0.0 0.0 - [bufdaemo
17 DL 0:00.00 0 127 0 0 16 - 0 0.0 0.0 - [vnlru]
18 DL 0:00.01 0 127 0 0 16 - 0 0.0 0.0 - [syncer]
30 DL 0:00.00 1 127 0 0 16 - 0 0.0 0.0 - [aiod1]
31 DL 0:00.00 1 127 0 0 16 - 0 0.0 0.0 - [aiod2]
32 DL 0:00.00 1 127 0 0 16 - 0 0.0 0.0 - [aiod3]
33 DL 0:00.00 1 127 0 0 16 - 0 0.0 0.0 - [aiod4]
No idea what any of that means though...
Thanks
but the output is not usable since the shell window cut the interesting parts off at the end. So make the window larger and then execute the command again.
Try to look for commands that consume a lot of %MEM, then paste their whole command tree.
Dont paste everything, only whats interesting.
(You can also paste the output into AI and let it filter the interesting things out before posting)
Thanks Cedrik,
And apologies, I should have realised that. I tned to work these things out by trial and error usually.
I did find a whole lot of entries like the below, not very high %mem at anywhere between .1 and .3 but a LOT of them.:
99972 I 0:00.13 127 127 0 70932 11076 - 6404 0.0 0.1 |-- /usr/local/bin/php /usr/local/etc/rc.newwanipv6 igc0 REQUEST
Is this potentially something to do with DHCP6? I think I have this enabled for my WAN connection but I have no idea if my ISP support it? I might try disabling to see if that resolves the issue.
Thanks again,
Mike
Think thats fixed it. I disabled DHCP6 on WAN
Memory usage isn't creeping up anymore. sat at just under 8% for the last hour.
Thanks for the help!
Hello,
no problem, glad you found it out.
It's a little weird DHCPv6 client spammed a lot of forks and consumed your RAM, maybe there was some unfortunate condition.
Maybe try to reboot and enable it again sometime to see if the same happens again.
If it does check your system log file (System: Log Files: General) what it does.