[Solved] OPNsense suddenly unreachable - How to debug (other issue)

[gothbert]

Dear all,

it randomly occurs that my OPNSense is suddenly not reachable any more. I am currently (still) on OPNsense 24.7.1-amd64 but had these issues say once every quarter for long.

It just occured ten minutes ago. I had to power the device down and back on. I read the thread with the same subject from today but it did not help. There is only the latest log showing entries since the last boot at 13:03 in /var/log/system. What I need are the logs from the previous run. Where have they gone? System | Settings | Logging, Tab "Local" is set to keep 31 log files.

Any help would be appreciated.

Best regards,
Boris

[doktornotor]

They have gone nowhere. They are in /var/log. Rotated and retained according to your settings.

Code: [Select]

# ls -la /var/log/system/
total 405
drwx------   2 root wheel     34 Aug 22 13:01 .
drwxr-xr-x  27 root wheel     42 Aug 20 08:26 ..
lrwxr-x---   1 root wheel     35 Aug 22 13:01 latest.log -> /var/log/system/system_20240822.log
-rw-------   1 root wheel  16053 Jul 23 23:37 system_20240723.log
-rw-------   1 root wheel  15880 Jul 24 23:37 system_20240724.log
-rw-------   1 root wheel  16512 Jul 25 23:37 system_20240725.log
-rw-------   1 root wheel  17310 Jul 26 23:37 system_20240726.log
-rw-------   1 root wheel  18025 Jul 27 23:37 system_20240727.log
-rw-------   1 root wheel  19380 Jul 28 23:37 system_20240728.log
-rw-------   1 root wheel  23648 Jul 29 23:39 system_20240729.log
-rw-------   1 root wheel  17510 Jul 30 23:37 system_20240730.log
-rw-------   1 root wheel  15723 Jul 31 23:37 system_20240731.log
-rw-------   1 root wheel  16634 Aug  1 23:37 system_20240801.log
-rw-------   1 root wheel  16059 Aug  2 23:37 system_20240802.log
-rw-------   1 root wheel  19690 Aug  3 23:37 system_20240803.log
-rw-------   1 root wheel  20682 Aug  4 23:49 system_20240804.log
-rw-------   1 root wheel  19866 Aug  5 23:37 system_20240805.log
-rw-------   1 root wheel  15168 Aug  6 23:37 system_20240806.log
-rw-------   1 root wheel  16392 Aug  7 23:37 system_20240807.log
-rw-------   1 root wheel  19368 Aug  8 23:37 system_20240808.log
-rw-------   1 root wheel  17314 Aug  9 23:41 system_20240809.log
-rw-------   1 root wheel 397859 Aug 10 23:45 system_20240810.log
-rw-------   1 root wheel 131689 Aug 11 23:46 system_20240811.log
-rw-------   1 root wheel  19785 Aug 12 23:57 system_20240812.log
-rw-------   1 root wheel  15010 Aug 13 23:46 system_20240813.log
-rw-------   1 root wheel 190857 Aug 14 23:35 system_20240814.log
-rw-------   1 root wheel  15588 Aug 15 23:35 system_20240815.log
-rw-------   1 root wheel  14807 Aug 16 23:35 system_20240816.log
-rw-------   1 root wheel 133861 Aug 17 23:47 system_20240817.log
-rw-------   1 root wheel  92127 Aug 18 23:58 system_20240818.log
-rw-------   1 root wheel  18280 Aug 19 23:42 system_20240819.log
-rw-------   1 root wheel  15359 Aug 20 23:42 system_20240820.log
-rw-------   1 root wheel  15168 Aug 21 23:42 system_20240821.log
-rw-------   1 root wheel 173904 Aug 22 13:28 system_20240822.log


As for the various question on how to filter / parse them, really do not have time for this. See the grep manpage.

[gothbert]

Actually, they are not there. That's why I am asking. What you showed is what I expected but the directory only contains the current log file:

Code: [Select]

# ls -la /var/log/system/
total 61
drwx------   2 root wheel   128 Aug 22 14:01 .
drwxr-xr-x  12 root wheel   960 Aug 22 13:10 ..
lrwxr-x---   1 root wheel    35 Aug 22 14:01 latest.log -> /var/log/system/system_20240822.log
-rw-------   1 root wheel 61245 Aug 22 13:35 system_20240822.log

[gothbert]

It just dawned to me: /var/log is on tmpfs and does not survive a power off/power on cycle.

[doktornotor]

Well, that won't work. You can either disable that feature (that should've been gone years ago IMO), or set up a syslog server somewhere to forward the logs to.

[chemlud]

System -> Settings -> Logging

Have you enabled local logging? You can choose the number of days to be saved...

[doktornotor]

Have you enabled local logging? You can choose the number of days to be saved...

He's got them on ramdisk (tmpfs). That is not very persistent on crash.

[chemlud]

That would be

System -> Settings -> Misc -> Section "Disk Memory Settings"

I think this is not enabled as tmpfs by default, or?

[doktornotor]

I think this is not enabled as tmpfs by default, or?

Definitely not.

[gothbert]

I must have enabled tmpfs for logging years ago to reduce the wearout on the SSD. I have disabled it under System | Settings | Miscellaneous. It requires a reboot, though, to become effective.

[chemlud]

I must have enabled tmpfs for logging years ago to reduce the wearout on the SSD. I have disabled it under System | Settings | Miscellaneous. It requires a reboot, though, to become effective.

...which makes perfectly sense to me, though...