OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • Archive »
  • 20.7 Legacy Series »
  • Swap used to 70%?
« previous next »
  • Print
Pages: [1]

Author Topic: Swap used to 70%?  (Read 6529 times)

PotatoCarl

  • Full Member
  • ***
  • Posts: 134
  • Karma: 5
    • View Profile
Swap used to 70%?
« on: September 10, 2020, 11:36:00 am »
Hi
I use an appliance from Deciso with my OPNSense. I am wondering why I get a swap use of about 70% (5GB).
I am pretty sure that it is not a very good idea to Swap on the SSD all the time, but I do not find a good overview, which program is actually swaping. Maybe I just overlooked it but would apreaciate a hint.
Cheers
Logged

siga75

  • Full Member
  • ***
  • Posts: 185
  • Karma: 11
    • View Profile
    • www.signorini.ch
Re: Swap used to 70%?
« Reply #1 on: September 10, 2020, 01:19:52 pm »
you don't want to know the program that has memory swapped out, you should find the process that eat all the RAM and makes other processes to swap out

top command gives you an overview of the memory used by processes, except shared segments, for which you can use "ipcs -a"

to have information on the process being swapped out

[root@myfw ~]# swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/gpt/swapfs   8388608   408476  7980132     5%
[root@myfw ~]# ps auxvw
USER      PID  %CPU %MEM     VSZ     RSS TT  STAT STARTED        TIME COMMAND           SL  RE PAGEIN LIM TSIZ
squid    7496   0.0  1.3 1588020  213728  -  S     3Sep20    58:40.26 (squid-1) --kid    0 127   3042   - 4472
squid   30228   0.0  0.0 1249468     648  -  Is    3Sep20     0:00.00 /usr/local/sbin/ 127 127     27   - 4472
root    16852   0.1  5.9 3443616  986580  -  Ss   Mon16      50:24.28 /usr/local/bin/s   0 127   2592   - 3904
www     57923   0.0  0.0  197984    3692  -  Ss    3Sep20     0:34.39 /usr/local/sbin/   0 127     26   - 1992
root     4326   0.0  0.0   39632      56  -  I     3Sep20     0:00.00 php-fpm: pool we 127 127      0   - 2532
www     14485   0.0  0.0   39632      56  -  I     3Sep20     0:00.00 php-fpm: pool ww 127 127      0   - 2532
root    44326   0.0  0.0   39632      56  -  I     3Sep20     0:00.00 php-fpm: pool we 127 127      0   - 2532
www     60404   0.0  0.0   39632      56  -  I     3Sep20     0:00.00 php-fpm: pool ww 127 127      0   - 2532
root    97642   0.0  0.0   39556    3904  -  Ss    3Sep20     0:22.54 php-fpm: master    0 127      0   - 2532
root    12647   0.0  0.5  104164   78140  -  I    Sun19       0:20.92 /usr/local/bin/p 127 127     11   - 2468
root    19873   0.0  0.3   82408   56468  -  I    Sun19       0:01.67 /usr/local/bin/p 127 127      0   - 2468
root    35072   0.0  0.4   90584   63668  -  I    Sun15       0:21.20 /usr/local/bin/p 127 127     78   - 2468
root    57400   0.0  0.0   39428     424  -  Is    3Sep20     0:00.03 /usr/local/bin/p 127 127     74   - 2468
root    63011   0.0  0.0   39428     440  -  Is    3Sep20     0:00.08 /usr/local/bin/p 127 127     28   - 2468
root    63827   0.0  0.3   75116   48852  -  I    Sun16       0:12.66 /usr/local/bin/p 127 127      0   - 2468
root    67420   0.0  0.2   59172   34284  -  I    Sun19       0:19.97 /usr/local/bin/p 127 127      0   - 2468
root    88293   0.0  0.4   89840   64328  -  I    Sun19       0:21.45 /usr/local/bin/p 127 127      3   - 2468
root    22284   0.0  0.0   13860     996  -  Ss    3Sep20     0:00.39 /sbin/devd         5 127     19   -  720
root        1   0.0  0.0    9588     152  -  ILs   3Sep20     0:00.54 /sbin/init --     52 127     31   -  648
dhcpd   88018   0.0  0.0   23408    2732  -  Ss    3Sep20     0:17.88 /usr/local/sbin/   0 127     62   - 2344
root    20077   0.0  0.0 1073960    3368  -  Ss    3Sep20     0:26.05 /usr/local/sbin/   0 127     19   -  636
root    52944   0.0  0.0 1070056    5004  0  I    13:02       0:00.01 /bin/csh         127 127     20   -  632
root    71275   0.0  0.0 1069752    5200  0  S    13:02       0:00.02 bash               0 127      5   -  660
root    43759   0.0  0.0 1071804    3188  -  Is    3Sep20     0:00.00 sshd: /usr/local 127 127     14   -  604
root    76191   0.0  0.0 1072348    7456  -  Ss   13:02       0:00.06 sshd: root@pts/0   0 127      0   -  604
www      9222   0.0  0.0  362960    1272  -  S     3Sep20     0:04.10 nginx: cache man   9 127      4   -  780
root    23327   0.0  0.0 1065424    4452  -  Ss    3Sep20     0:05.72 /usr/local/sbin/   6 127     24   -  524
root    45344   0.0  0.0  362924    1000  -  Is    3Sep20     0:02.88 nginx: master pr  52 127      0   -  780
root    61518   0.0  0.0 1071808    3292  -  Ss    3Sep20     0:04.76 /usr/local/sbin/   9 127      5   -  524
www     90659   0.0  0.0  360732    7136  -  S     3Sep20     2:21.98 nginx: worker pr  18 127    226   -  780
redis   84171   0.0  0.0   15356    1620  -  Ss    3Sep20     2:56.39 redis-server: /u   0 127     11   -  828
unbound 35027   0.0  0.2   74604   29580  -  Ss    3Sep20    17:52.12 /usr/local/sbin/  18 127   3159   -  720
root    36429   0.0  0.0 1061540    1300  -  Ss    3Sep20     1:58.05 /bin/sh /var/db/   8 127      0   -  188
root    14765   0.0  0.0 1061388    3240  0  Is   13:02       0:00.00 /bin/sh /usr/loc 127 127      0   -  188
squid   11764   0.0  0.0 1066996    3264  -  S    Sat00       0:11.69 (pinger) (pinger   0 127      0   -   92
squid   11860   0.0  0.0 1073320    7224  -  S    Mon00       0:07.66 (pinger) (pinger   0 127     29   -   92
squid   35898   0.0  0.0 1067160    3324  -  S    Sun00       0:09.81 (pinger) (pinger   0 127      4   -   92
squid   41761   0.0  0.0 1067140    7452  -  S    Mon00       0:07.96 (pinger) (pinger   0 127      0   -   92
squid   42796   0.0  0.0 1067184    3264  -  S     3Sep20     0:13.89 (pinger) (pinger   0 127      4   -   92
squid   48766   0.0  0.1 1067128    8696  -  S    Tue00       0:06.01 (pinger) (pinger   0 127      0   -   92
squid   54168   0.0  0.1 1067128    8696  -  S    00:00       0:01.41 (pinger) (pinger   0 127      0   -   92
squid   55264   0.0  0.1 1067164    8700  -  S    Wed00       0:03.71 (pinger) (pinger   0 127      0   -   92
squid   74367   0.0  0.0 1067184    3268  -  S    Fri00       0:13.59 (pinger) (pinger   0 127      0   -   92
squid   38597   0.0  0.0 1072044    6808  -  I    Mon00       0:01.81 (security_file_c 127 127      0   -  116
squid   90190   0.0  0.0 1065660    6496  -  I    Mon00       0:00.04 (security_file_c 127 127      2   -  116
root    62767   0.0  0.0 1060672    1360  -  Is    3Sep20     0:00.00 /usr/sbin/moused 127 127      0   -   76
root    35181   0.0  0.0 1061384    3064  0  R+   13:15       0:00.00 ps auxvw         127   0      0   -   72
root    31610   0.0  0.0 1061276    2020  -  I    13:00       0:00.00 cron: running jo 127 127      0   -   76
root    44857   0.0  0.0 1061276    1068  -  Is    3Sep20     0:10.65 /usr/sbin/cron -  47 127      0   -   76
root    95276   0.0  0.0 1060708    1368 u0  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      1   -   64
root    20043   0.0  0.0 1060708    1372 v0  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      2   -   64
root    29063   0.0  0.0 1060708    1368 v1  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    99709   0.0  0.0 1060708    1368 v2  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    11383   0.0  0.0 1060708    1368 v3  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    66073   0.0  0.0 1060708    1368 v4  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    33776   0.0  0.0 1060708    1372 v5  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    88360   0.0  0.0 1060708    1368 v6  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
root    19616   0.0  0.0 1060708    1368 v7  Is+   3Sep20     0:00.00 /usr/libexec/get 127 127      0   -   64
nobody  23662   0.0  0.0 1060552     248  -  S     3Sep20     0:07.21 /usr/local/bin/s   1 127      1   -   60
root    60553   0.0  0.0 1061692    1680  -  Ss    3Sep20     0:22.95 /usr/local/sbin/   0 127      1   -   60
root    80167   0.0  0.0 1060668    1388  -  Is    3Sep20     0:00.00 daemon: /usr/loc 127 127      0   -   56
root     7913   0.0  0.0 1060504    2476  -  SC   13:15       0:00.00 sleep 60           8   8      0   -   52
root    22429   0.0  0.0   15708    3276  -  Ss    3Sep20     0:04.11 /usr/local/sbin/   3 127     48   -  384
zabbix   8174   0.0  0.0   23836    3772  -  S     3Sep20     0:39.14 zabbix_agentd: l   1 127     11   -  304
zabbix  69630   0.0  0.0   24056    3824  -  S     3Sep20     0:58.02 zabbix_agentd: c   0 127      2   -  304
zabbix  81941   0.0  0.0   24116    3480  -  S     3Sep20     0:36.61 zabbix_agentd: a   0 127     12   -  304
zabbix  85987   0.0  0.0   23900    1836  -  S     3Sep20     0:38.47 zabbix_agentd: l   1 127      9   -  304
zabbix  99597   0.0  0.0   24056       0  -  IW   -           0:00.00 /usr/local/sbin/ 127 127      0   -  304
root    14694   0.0  0.0   15188    2692  -  S     3Sep20     0:20.04 /usr/local/sbin/   0 127      0   -  208
root    45792   0.0  0.0   17616    5172  -  S     3Sep20     0:29.13 /usr/local/sbin/   0 127     21   -  208
c_icap  21663   0.0  0.0   22808    2812  -  S    00:00       0:01.57 /usr/local/bin/c   0 127      0   -   88
c_icap  28267   0.0  0.0   21644    2464  -  Ss    3Sep20     0:20.35 /usr/local/bin/c   0 127     73   -   88
c_icap  46587   0.0  0.0   41260    3988  -  S    00:00       0:01.94 /usr/local/bin/c   0 127      0   -   88
clamav  72579   0.0  6.8 1200404 1127384  -  Is    3Sep20    18:23.29 /usr/local/sbin/ 127 127  27739   -   64
_flowd  44621   0.0  0.0   11452    1556  -  Ss    3Sep20     0:16.02 flowd: net (flow   1 127     27   -   72
root    88564   0.0  0.0   10992     288  -  Is    3Sep20     0:00.00 flowd: monitor ( 127 127     16   -   72
clamav  76927   0.0  0.0   27772    8284  -  Is    3Sep20     0:09.15 /usr/local/bin/f 127 127    370   -   36
root    40706   0.0  0.0   13944    1488  -  Is    3Sep20     0:59.92 /usr/local/bin/d 127 127      4   -   12
root      110   0.0  0.1   98228   19908  -  I     3Sep20     2:02.45 /usr/local/bin/p  53 127    113   -    4
root    10197   0.0  0.0   33940    6488  -  Ss    3Sep20    35:55.09 /usr/local/sbin/   0 127     60   -    4
root    19612   0.0  0.2   42596   33048  -  Is   13:00       0:00.45 /usr/local/bin/p 127 127      0   -    4
root    30558   0.0  0.1   31788   14568  -  Ss    3Sep20   311:12.30 /usr/local/bin/p   0 127     61   -    4
root    37577   0.0  0.0   21472    6872  -  S     3Sep20     0:58.74 /usr/local/bin/p   0 127    229   -    4
root    72298   0.0  0.0   18232       0  -  IW   -           0:00.00 /usr/local/sbin/ 127 127      0   -    4
root    85124   0.0  0.0   33004       0  -  IWs  -           0:00.00 /usr/local/bin/p 127 127      0   -    4
root        0   0.0  0.0       0     560  -  DLs   3Sep20   121:22.01 [kernel]           9 127      0   -    0
root        2   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [crypto]         127 127      0   -    0
root        3   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [crypto returns  127 127      0   -    0
root        4   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [crypto returns  127 127      0   -    0
root        5   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [crypto returns  127 127      0   -    0
root        6   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [crypto returns  127 127      0   -    0
root        7   0.0  0.0       0      32  -  DL    3Sep20     0:00.00 [cam]            127 127      0   -    0
root        8   0.0  0.0       0      16  -  DL    3Sep20     0:00.16 [soaiod1]         29 127      0   -    0
root        9   0.0  0.0       0      16  -  DL    3Sep20     0:00.16 [soaiod2]         28 127      0   -    0
root       10   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [audit]          127 127      0   -    0
root       11 400.0  0.0       0      64  -  RNL   3Sep20 39488:27.95 [idle]           127 127      0   -    0
root       12   0.0  0.0       0     240  -  WL    3Sep20   145:01.17 [intr]           127 127      0   -    0
root       13   0.0  0.0       0      48  -  DL    3Sep20     0:00.01 [geom]           127 127      0   -    0
root       14   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [sequencer 00]   127 127      0   -    0
root       15   0.0  0.0       0      80  -  DL    3Sep20     0:25.17 [usb]            127 127      0   -    0
root       16   0.0  0.0       0      16  -  DL    3Sep20     0:00.16 [soaiod3]          2 127      0   -    0
root       17   0.0  0.0       0      16  -  DL    3Sep20     0:00.16 [soaiod4]          3 127      0   -    0
root       18   0.0  0.0       0      16  -  DL    3Sep20     0:00.00 [sctp_iterator]  127 127      0   -    0
root       19   0.0  0.0       0      16  -  DL    3Sep20     3:24.54 [pf purge]         0 127      0   -    0
root       20   0.0  0.0       0      16  -  DL    3Sep20     5:33.70 [rand_harvestq]    0 127      0   -    0
root       21   0.0  0.0       0      16  -  DL    3Sep20     0:07.08 [acpi_thermal]     7 127      0   -    0
root     3822   0.0  0.0       0      16  -  DL    3Sep20     0:05.86 [vnlru]            0 127      0   -    0
root    20454   0.0  0.0       0      64  -  DL    3Sep20     0:03.24 [ng_queue]        11 127      0   -    0
root    32263   0.0  0.0       0     144  -  DL    3Sep20     1:35.75 [bufdaemon]        0 127      0   -    0
root    48050   0.0  0.0       0      48  -  DL    3Sep20     2:27.99 [pagedaemon]       0 127      0   -    0
root    87973   0.0  0.0       0      16  -  DL    3Sep20     0:01.01 [vmdaemon]       127 127      0   -    0
root    91598   0.0  0.0       0      16  -  DL    3Sep20     6:20.25 [syncer]           0 127      0   -    0
Logged
https://www.signorini.ch
Protectli Pfsense Mi7500L6 Intel 7Th Gen Core I7 7500U 16Gb Ddr4 Ram
512Gb Msata Ssd
6 X Intel Gigabit Ethernet

Vilhonator

  • Full Member
  • ***
  • Posts: 245
  • Karma: 13
    • View Profile
Re: Swap used to 70%?
« Reply #2 on: September 10, 2020, 02:13:11 pm »
There also could be more simpler reason for huge swap usage, have you rebooted your opnsense device recently?

Most common reason why people face huge swap usage is because they don't reboot their systems and keep them up and running (RAM does not empty itself AT ALL as long as system is up and running.)

If your opnsense has around 4GB RAM on it, it is possible that because you haven't rebooted it for a while, your system has just ran out of RAM and now uses SWAP instead. Simple reboot should fix that.

I would highly recommend setting automated reboot (shown on the image), that way you don't have to remember anything other than take backups before or after reboot, and that internet doesn't work at 00:00 AM local time first day of every month (if you schedule it like I do, and still panic when I get disconnected in middle of a raid boss fight every month -_-)
Logged

siga75

  • Full Member
  • ***
  • Posts: 185
  • Karma: 11
    • View Profile
    • www.signorini.ch
Re: Swap used to 70%?
« Reply #3 on: September 10, 2020, 03:59:16 pm »
Quote from: Vilhonator on September 10, 2020, 02:13:11 pm
There also could be more simpler reason for huge swap usage, have you rebooted your opnsense device recently?

Most common reason why people face huge swap usage is because they don't reboot their systems and keep them up and running (RAM does not empty itself AT ALL as long as system is up and running.)

If your opnsense has around 4GB RAM on it, it is possible that because you haven't rebooted it for a while, your system has just ran out of RAM and now uses SWAP instead. Simple reboot should fix that.

I would highly recommend setting automated reboot (shown on the image), that way you don't have to remember anything other than take backups before or after reboot, and that internet doesn't work at 00:00 AM local time first day of every month (if you schedule it like I do, and still panic when I get disconnected in middle of a raid boss fight every month -_-)

This is not true at all! RAM is free when the processes release it or when they finish

If the system used the swap is because it had a lack of RAM at some point, then it swapped. After that, even if you have free RAM, the system does not put back the swap in it just because it is non sense. If a process will need to access that part of swapped out memory then a that point a swap in happens.

If you see RAM almost always at 100% or close it's just because, like every unix systems, freeBSD is smart and use all the needed and available memory for file system caching, the biggest part of the FS caching (the non dirty pages) can be released in an instant, if needed by a process.

Of course a reboot will free up the swap if you don't like to see swap usage, but this is not needed, it's just cosmetics
Logged
https://www.signorini.ch
Protectli Pfsense Mi7500L6 Intel 7Th Gen Core I7 7500U 16Gb Ddr4 Ram
512Gb Msata Ssd
6 X Intel Gigabit Ethernet

Vilhonator

  • Full Member
  • ***
  • Posts: 245
  • Karma: 13
    • View Profile
Re: Swap used to 70%?
« Reply #4 on: September 10, 2020, 05:52:18 pm »
Quote from: siga75 on September 10, 2020, 03:59:16 pm
This is not true at all! RAM is free when the processes release it or when they finish

Oh so service or process which has 4GBs reserved to RAM or SWAP file can release 8GBs of ram used by 20 different services or processes at that time? Yay
Logged

siga75

  • Full Member
  • ***
  • Posts: 185
  • Karma: 11
    • View Profile
    • www.signorini.ch
Re: Swap used to 70%?
« Reply #5 on: September 10, 2020, 06:07:08 pm »
Quote from: Vilhonator on September 10, 2020, 05:52:18 pm
Quote from: siga75 on September 10, 2020, 03:59:16 pm
This is not true at all! RAM is free when the processes release it or when they finish

Oh so service or process which has 4GBs reserved to RAM or SWAP file can release 8GBs of ram used by 20 different services or processes at that time? Yay

You just don't know what you are talking about. I will not even reply.
Good luck with your "Windows Style" periodic reboot
Logged
https://www.signorini.ch
Protectli Pfsense Mi7500L6 Intel 7Th Gen Core I7 7500U 16Gb Ddr4 Ram
512Gb Msata Ssd
6 X Intel Gigabit Ethernet

PotatoCarl

  • Full Member
  • ***
  • Posts: 134
  • Karma: 5
    • View Profile
Re: Swap used to 70%?
« Reply #6 on: September 11, 2020, 09:04:31 am »
Hi
so when using "ps auxvw" this is little helpful, as the command line is cut so I cannot see the process.

So when I use
Code: [Select]
ps auxwm I get a much clearer picture. Unfortunately, the "m" option does not, as it is supposed to do, sort by memory usage.

Going manually through it, "clamd" uses about 40.6% (!), maltrail with several processes 11.6 and 3x 12.2%

I use clamd with i-cap for squid, but I am amazed about the memory usage. As I am using on other servers clamd (among other virus scanners) for mail scanning, this seems to be a lot of memory to use.
I removed the plugin, however, it is interestingly still shown in the "ps" output.
The appliance only has 4GB ram, however, as it runs headless I am assuming that should be more than enough.
After a reboot the memory usage is more to normal "Swap=0".

Thanks.
Logged

Tubs

  • Full Member
  • ***
  • Posts: 100
  • Karma: 3
    • View Profile
Re: Swap used to 70%?
« Reply #7 on: September 13, 2020, 03:37:51 am »
I have the same issue sind 20.7.x.

Before the system was running fine with 8 GB RAM installed and plenty of free RAM. No other change beside upgrade to 20.7.x. In my case after a while Sensei is running out of memory, stops automatically and is getting disabled. After Sensei stopped all is fine. But if Sensei is cause or effect I cannot say.

Code: [Select]
Sensei has detected a problem during operation and has shut down Sensei services in order to prevent a network outage.

It is because we detected high SWAP usage 36 % ( 2.95GB / 8GB )

If you think this is something we should have a look, just click here to let us know about the details and we will investigate this further.

You can re-enable the services from Status page.
Logged

GreenMatter

  • Full Member
  • ***
  • Posts: 166
  • Karma: 1
    • View Profile
Re: Swap used to 70%?
« Reply #8 on: November 24, 2020, 12:45:20 pm »
Quote from: Tubs on September 13, 2020, 03:37:51 am
I have the same issue sind 20.7.x.

Before the system was running fine with 8 GB RAM installed and plenty of free RAM. No other change beside upgrade to 20.7.x. In my case after a while Sensei is running out of memory, stops automatically and is getting disabled. After Sensei stopped all is fine. But if Sensei is cause or effect I cannot say.

Code: [Select]
Sensei has detected a problem during operation and has shut down Sensei services in order to prevent a network outage.

It is because we detected high SWAP usage 36 % ( 2.95GB / 8GB )

If you think this is something we should have a look, just click here to let us know about the details and we will investigate this further.

You can re-enable the services from Status page.


I keep getting same as above - SWAP being used over 30% and Sensei gets disabled. Prior to 20.7, SWAP almost never been used. And 8GB of RAM is utilized at about 60% or less as average maximum. Do I need to increase RAM or only change settings of "something"?
Logged
OPNsense on:
Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (4 cores)
8 GB RAM
50 GB HDD
and plenty of vlans ;-)

Tubs

  • Full Member
  • ***
  • Posts: 100
  • Karma: 3
    • View Profile
Re: Swap used to 70%?
« Reply #9 on: November 29, 2020, 02:04:17 pm »
Since 20.7.4 it is better again. Still more swap is used than before. But no service is stopping any more.
Logged
  • Print
Pages: [1]
« previous next »
  • OPNsense Forum »
  • Archive »
  • 20.7 Legacy Series »
  • Swap used to 70%?
 

OPNsense is an OSS project © Deciso B.V. 2015 - 2024 All rights reserved
  • SMF 2.0.19 | SMF © 2021, Simple Machines
    Privacy Policy     | XHTML | RSS | WAP2