Zenarmor was causing this, even when running in bypassed mode. Stopped the Engine, Reporting Database and Clouds Agent didn“t help, after uninstalling Zenarmor packet drops stopped.
Found this in the logs that made me suspicious about the Zenarmor:
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1993"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1994"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1995"] action allowed zenarmor.check-remote-elastic for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1996"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1997"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1998"] action allowed zenarmor.aliases for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1999"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2000"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2001"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2002"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2003"] action allowed zenarmor.aliases for user root
<38>1 2025-02-03T05:22:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2004"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:22:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2005"] action allowed zenarmor.check-health for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2006"] action allowed zenarmor.numberofdevice for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2007"] action allowed zenarmor.license-details for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2008"] action allowed zenarmor.license-details for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2009"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2010"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2011"] action allowed ids.list.rulemetadata for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2012"] action allowed zenarmor.engine-version for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2013"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2014"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2015"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2016"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2017"] action allowed zenarmor.aliases for user root
Found this in the logs that made me suspicious about the Zenarmor:
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1993"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1994"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1995"] action allowed zenarmor.check-remote-elastic for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1996"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1997"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:20:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1998"] action allowed zenarmor.aliases for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="1999"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2000"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2001"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2002"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:21:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2003"] action allowed zenarmor.aliases for user root
<38>1 2025-02-03T05:22:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2004"] action allowed zenarmor.periodicals for user nobody
<38>1 2025-02-03T05:22:00+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2005"] action allowed zenarmor.check-health for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2006"] action allowed zenarmor.numberofdevice for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2007"] action allowed zenarmor.license-details for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2008"] action allowed zenarmor.license-details for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2009"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:04+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2010"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2011"] action allowed ids.list.rulemetadata for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2012"] action allowed zenarmor.engine-version for user root
<38>1 2025-02-03T05:22:05+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2013"] action allowed zenarmor.service for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2014"] action allowed zenarmor.every-minute-check for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2015"] action allowed zenarmor.scheduled-service-control for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2016"] action allowed zenarmor.userenrich for user root
<38>1 2025-02-03T05:22:21+01:00 nnn.mydomain.zzz configd.py 15939 - [meta sequenceId="2017"] action allowed zenarmor.aliases for user root