Hi All,
since a few weeks I noticed that the Crowdsec daemon is stopping / crashing at 1am (which should be UTC midnight).
I don't see anything in the crowdsec logs.
I'm not sure if this is happening since OPNsense 24 or if my IPv6 changes added additional load on the server. I would say the LAPI server is gone as I can see that the bounce is still trying to communicate.
Could it be that the local LAPI server is at the capacity limit?
Service is looking normal after starting it again.
Thank for your help
Timmi
I have the Same problem.
I couldn't see anything in the Logs, where do you see when the connection is broken? @TimmiORG
Hi,
I see in /var/log/crowdsec/crowdsec-firewall-bouncer.log
time="08-03-2024 01:00:52" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 172.28.52.65:8080: i/o timeout"
time="08-03-2024 01:00:52" level=error msg="Get \"http://192.168.1.1:8080/v1/decisions/stream?\": dial tcp 192.168.1.1:8080: i/o timeout"
ok on my side its not the same but similar
time="08-03-2024 07:20:46" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="08-03-2024 07:20:46" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
@TimmiORG what is the actual output from
tail /var/log/crowdsec/crowdsec-firewall-bouncer.log ?
like this
time="08-03-2024 12:45:32" level=info msg="1 decision added"
This is what is shown if the system is running.
Hi,
it's not the bouncer logs that you should read, but crowdsec.log
Is there anything that points to a service failure?
I know.
As I wrote I don't see anything specific in the crowdsec.log at that time. Just no logs anymore at some point.
Only the bouncer log is showing that the LAPI is not available as I wrote.
Quote from: mmetc on March 08, 2024, 12:47:14 PM
Hi,
it's not the bouncer logs that you should read, but crowdsec.log
Is there anything that points to a service failure?
ok got it.
I think this could be the guilty guy?
time="2024-03-08T11:36:42+01:00" level=warning msg="sqlite is not using WAL mode, LAPI might become unresponsive when inserting the community blocklist"
Not for me as WAL mode is enabled.
I also don't receive the warning.
the best thing is I have that already activated ;)
I have created a Monit test to restart the service is it is not running.
So the service should be back within two minutes.
Quote from: TimmiORG on March 08, 2024, 04:10:11 PM
I have created a Monit test to restart the service is it is not running.
So the service should be back within two minutes.
Interesting, could you explain how you're doing that?
Sure, I assume Monit is running already.
Service Test Settings:
Name: Crowdsec_Service
Condition: failed host 127.0.0.1 port 8080 type tcp
Action: Restart
Service Settings:
Enable service checks: yes
Name: Crowdsec
Type: Process
PID File: /var/run/crowdsec.pud
Start: /usr/sbin/service crowdsec start
Stop: /usr/sbin/service crowdsec stop
Tests: Crowdsec_Service
Depends: Nothing selected
Description: Check that Crowdsec is running
Just chiming in that I am also seeing this on my end as well. Crowdsec goes down every night now it seems. Going to look in the Monit advice from the prior posts in the meantime.
v24.1.3_1
tail /var/log/crowdsec/crowdsec.log
time="2024-03-08T12:14:31-05:00" level=info msg="Adding file /var/log/audit/latest.log to datasources" type=file
time="2024-03-08T12:14:31-05:00" level=info msg="Force add watch on /var/log/lighttpd" type=file
time="2024-03-08T12:14:31-05:00" level=info msg="Adding file /var/log/lighttpd/latest.log to datasources" type=file
time="2024-03-08T12:14:31-05:00" level=info msg="Force add watch on /var/log/filter" type=file
time="2024-03-08T12:14:31-05:00" level=info msg="Adding file /var/log/filter/latest.log to datasources" type=file
time="2024-03-08T12:14:31-05:00" level=info msg="Starting processing data"
time="2024-03-08T12:14:34-05:00" level=info msg="capi/community-blocklist : 0 explicit deletions"
time="2024-03-08T12:14:34-05:00" level=warning msg="sqlite is not using WAL mode, LAPI might become unresponsive when inserting the community blocklist"
time="2024-03-08T12:14:34-05:00" level=info msg="crowdsecurity/community-blocklist : added 15000 entries, deleted 14449 entries (alert:453)"
time="2024-03-08T12:14:34-05:00" level=info msg="Start pull from CrowdSec Central API (interval: 1h56m16s once, then 2h0m0s)"
Add me to the list, too. Preemptive thanks to whomever figures out a fix!
Mine isn't having this problem.
p.s. "level=warning msg="sqlite is not using WAL mode, LAPI might become unresponsive when inserting the community blocklist" seems to be only a warning. I used to get them and just set to WAL mode and the warning noise went away. That is what is for, telling you it will continue working but has a suggestion to improve.
This last snippet has no trace of a problem. So the question is why you think it is not running. Or more importantly, please keep looking in that log for other clues.
I can see the dashboard service status every day turn to a red play button. upon clicking it it starts back up and runs fine for that day.
not sure what logs i can also check, but like others have said, i cant seem to find it crashing in any logs. it just turns off until i start it again.
/var/log/crowdsec/crowdsec.log
time="2024-03-08T01:19:14-05:00" level=error msg="Failed to fetch network for 194.26.135.250 : the MaxMind DB file's data section contains bad data (float 64 size of 19)" id=morning-snow method=IpToRange name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:19:34-05:00" level=error msg="Unable to enrich ip '167.94.145.90'" id=morning-snow method=GeoIpASN name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:19:34-05:00" level=error msg="Failed to fetch network for 167.94.145.90 : unexpected type when decoding string: 79" id=morning-snow method=IpToRange name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:19:51-05:00" level=error msg="Unable to enrich ip '109.205.213.22'" id=morning-snow method=GeoIpASN name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:19:51-05:00" level=error msg="Failed to fetch network for 109.205.213.22 : the MaxMind DB file's data section contains bad data (float 64 size of 20)" id=morning-snow method=IpToRange name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:20:06-05:00" level=error msg="Unable to enrich ip '109.205.213.22'" id=morning-snow method=GeoIpASN name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T01:20:06-05:00" level=error msg="Failed to fetch network for 109.205.213.22 : the MaxMind DB file's data section contains bad data (float 64 size of 20)" id=morning-snow method=IpToRange name=crowdsecurity/geoip-enrich stage=s02-enrich
time="2024-03-08T12:14:30-05:00" level=warning msg="You are using sqlite without WAL, this can have a performance impact. If you do not store the database in a network share, set db_config.use_wal to true. Set explicitly to false to disable this warning."
time="2024-03-08T12:14:30-05:00" level=info msg="Enabled feature flags: <none>"
time="2024-03-08T12:14:30-05:00" level=info msg="Crowdsec v1.6.0-freebsd-4b8e6cd7"
time="2024-03-08T12:14:30-05:00" level=info msg="Loading prometheus collectors"
time="2024-03-08T12:14:30-05:00" level=info msg="Loading CAPI manager"
time="2024-03-08T12:14:30-05:00" level=info msg="flushed 6/33 alerts because they were created 7d ago or more"
time="2024-03-08T12:14:31-05:00" level=info msg="CAPI manager configured successfully"
time="2024-03-08T12:14:31-05:00" level=error msg="Machine is not enrolled in the console, can't synchronize with the console"
time="2024-03-08T12:14:31-05:00" level=info msg="Start push to CrowdSec Central API (interval: 11s once, then 10s)"
time="2024-03-08T12:14:31-05:00" level=info msg="CrowdSec Local API listening on 127.0.0.1:8080"
time="2024-03-08T12:14:31-05:00" level=info msg="Start sending metrics to CrowdSec Central API (interval: 17m52s once, then 30m0s)"
time="2024-03-08T12:14:31-05:00" level=info msg="capi metrics: sending"
time="2024-03-08T12:14:31-05:00" level=info msg="Loading grok library /usr/local/etc/crowdsec/patterns"
time="2024-03-08T12:14:31-05:00" level=info msg="Starting community-blocklist update"
/var/log/crowdsec/crowdsec_api.log
time="2024-03-08T01:19:21-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 01:19:21 EST] \"GET /v1/decisions/stream HTTP/1.1 200 19.186703ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T01:19:31-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 01:19:31 EST] \"GET /v1/decisions/stream HTTP/1.1 200 20.377403ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T01:19:41-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 01:19:41 EST] \"GET /v1/decisions/stream HTTP/1.1 200 19.258695ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T01:19:51-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 01:19:51 EST] \"GET /v1/decisions/stream HTTP/1.1 200 39.013967ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T01:20:01-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 01:20:01 EST] \"GET /v1/decisions/stream HTTP/1.1 200 25.659197ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T12:14:31-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 12:14:31 EST] \"POST /v1/watchers/login HTTP/1.1 200 54.670453ms \"crowdsec/v1.6.0-freebsd-4b8e6cd7\" \""
time="2024-03-08T12:14:45-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 12:14:45 EST] \"GET /v1/decisions/stream HTTP/1.1 200 224.060551ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T12:14:45-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 12:14:45 EST] \"GET /v1/decisions/stream HTTP/1.1 200 15.971222ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
time="2024-03-08T12:14:50-05:00" level=info msg="127.0.0.1 - [Fri, 08 Mar 2024 12:14:50 EST] \"GET /v1/decisions/stream HTTP/1.1 200 14.849763ms \"crowdsec-firewall-bouncer/v0.0.28-freebsd-af6e7e2\" \""
I've been seeing this as well. I thought it coorisponded with my CRON job that runs, "Update and reload firewall aliases" every night at 1:07am, but maybe it has nothing to do with that?
me too, before 24.2 and crowdsec 1.6 it was rock stable
now every day , dashboard show red and log show not anything , restart it runs smooth.
I've had to spend most of the weekend fixing my network for other reasons.
Those error messages seem pretty serious and seems MaxMid's database is in a different to the expected. As to what changed would be a guess. Can be either maxmind or crowdsec.
You could try disabling the enrich part whilst the problem is investigated. It sure looks like needs reporting also on their side, in case this forum isn't monitored much.
Quote from: cookiemonster on March 11, 2024, 02:43:41 PM
I've had to spend most of the weekend fixing my network for other reasons.
Those error messages seem pretty serious and seems MaxMid's database is in a different to the expected. As to what changed would be a guess. Can be either maxmind or crowdsec.
Hi, I'm the author of the opnsense plugin. A new version of the geoip database had issues with the current crowdsec and we reverted to the older version. Hub upgrade (manually or from cron) fixes it, and I don't think it could crash the service. I am looking into the issue. Thanks!
I'm also having this issue. I received an email from Maxmind yesterday stating they would be switching to R2 presigned URLs for all DBs, as of May 1st, and that it is a potential breaking change. Not sure if this is related to the issue we are facing but I figured I would mention it.
I tried running "cscli hub upgrade --force" on both of my routers and they fail on the "crowdsecurity/geoip-enrich" list.
Quote from: whezzel on March 12, 2024, 10:28:35 PM
I'm also having this issue. I received an email from Maxmind yesterday stating they would be switching to R2 presigned URLs for all DBs, as of May 1st, and that it is a potential breaking change. Not sure if this is related to the issue we are facing but I figured I would mention it.
I tried running "cscli hub upgrade --force" on both of my routers and they fail on the "crowdsecurity/geoip-enrich" list.
Did you run "cscli hub update" first?
I could not replicate the issue, but it would help if you ran "cscli support dump" and send the resulting file to support@crowdsec.net
Thanks!
Since 6d it did not crash.
Subscribed, since I regularly have crowdsec stop - for whatever reason and I cannot tell at what time. Have sent the crowdsec-support file, but I doubt that it reveals much (e.g. there is no crashdump in there).
I never noticed this - just to add a data point - but experience a different way of crowdsec to occasionally stop:
https://github.com/crowdsecurity/crowdsec/issues/2902
I meant: crowdsec sometimes stops - I cannot even tell at what time, much less what is the cause.
Log rotation and a resulting crash might well be it, however I just have reset my log files and that did not cause crowdsec to stop.
Yes crowdsec would inappropriately raise an error if a watched file disappears immediately after the initial directory scan
This will be corrected for 1.6.1, but I'm not sure how often it occurs.
More generally, a process exit by crowdsec could be due to CAPI being unavailable for a long time or other issues.
On the linux package any transient exit/crash is not a problem, expect for the possible underlying bug, since the process is restarted immediately by systemd (or docker). For freebsd there is no - afaik - general consensus on how to restart crashed processes.
Monit is a good solution but it's not available on freebsd by default or in pfsense. I tried simply adding a restart option to the sbin/daemon wrapper, it's not working as expected but I'd prefer the solution should be the same for the three platforms.
If someone is using monit to restart crowdsec, can you share that part of configuration?
Thanks
(https://forum.opnsense.org/index.php?action=dlattach;topic=39318.0;attach=33728;image)
(https://forum.opnsense.org/index.php?action=dlattach;topic=39318.0;attach=33730;image)
Hello,
Thanks for sending logs and configurations, we fixed some issue for the upcoming 1.6.1 and are looking at other possible causes.
In the meantime, we have a version of the base crowdsec package that restarts the service correctly when it fails.
You can find it at https://github.com/crowdsecurity/plugins/releases/tag/crowdsec-1.6.0_3
Let us know if it helps and thanks for testing,
Marco
Hello,
I think I have the same problem.
The service is stopped, I try to start it, for a few seconds the service icon is green but always returns to red.
I uninstalled - restarted OPNsense - intall Crowdsec, the problem is still there.
Note: I have had the problem for some time.
# tail /var/log/crowdsec/crowdsec-firewall-bouncer.log
time="28-04-2024 22:22:37" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:23:07" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:23:07" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:23:37" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:23:37" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:24:07" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:24:07" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:24:37" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:24:37" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:25:07" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:25:07" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:25:37" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 127.0.0.1:8080: i/o timeout"
time="28-04-2024 22:25:37" level=error msg="Get \"http://127.0.0.1:8080/v1/decisions/stream?\": dial tcp 127.0.0.1:8080: i/o timeout"
# tail /var/log/crowdsec/crowdsec.log
time="2024-04-28T22:23:30+02:00" level=warning msg="You are using sqlite without WAL, this can have a performance impact. If you do not store the database in a network share, set db_config.use_wal to true. Set explicitly to false to disable this warning."
time="2024-04-28T22:23:30+02:00" level=info msg="Enabled feature flags: <none>"
time="2024-04-28T22:23:30+02:00" level=info msg="Crowdsec v1.6.0-freebsd-4b8e6cd7"
time="2024-04-28T22:23:30+02:00" level=info msg="Loading prometheus collectors"
time="2024-04-28T22:23:31+02:00" level=info msg="Loading CAPI manager"
time="2024-04-28T22:23:32+02:00" level=info msg="CAPI manager configured successfully"
time="2024-04-28T22:23:32+02:00" level=error msg="Machine is not enrolled in the console, can't synchronize with the console"
time="2024-04-28T22:23:32+02:00" level=info msg="CrowdSec Local API listening on 127.0.0.1:8080"
time="2024-04-28T22:23:32+02:00" level=info msg="Start sending metrics to CrowdSec Central API (interval: 23m2s once, then 30m0s)"
time="2024-04-28T22:23:32+02:00" level=info msg="Start push to CrowdSec Central API (interval: 3s once, then 10s)"
time="2024-04-28T22:23:32+02:00" level=info msg="capi metrics: sending"
time="2024-04-28T22:23:32+02:00" level=info msg="last CAPI pull is newer than 1h30, skip."
time="2024-04-28T22:23:32+02:00" level=info msg="Start pull from CrowdSec Central API (interval: 2h1m51s once, then 2h0m0s)"
time="2024-04-28T22:23:32+02:00" level=info msg="Loading grok library /usr/local/etc/crowdsec/patterns"
time="2024-04-28T22:23:34+02:00" level=info msg="Loading enrich plugins"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'GeoIpCity'"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'GeoIpASN'"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'IpToRange'"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'reverse_dns'"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'ParseDate'"
time="2024-04-28T22:23:34+02:00" level=info msg="Successfully registered enricher 'UnmarshalJSON'"
time="2024-04-28T22:23:34+02:00" level=info msg="Loading parsers from 6 files"
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 2 parser nodes" file=/usr/local/etc/crowdsec/parsers/s00-raw/syslog-logs.yaml stage=s00-raw
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 1 parser nodes" file=/usr/local/etc/crowdsec/parsers/s01-parse/opnsense-gui-logs.yaml stage=s01-parse
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 2 parser nodes" file=/usr/local/etc/crowdsec/parsers/s01-parse/pf-logs.yaml stage=s01-parse
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 1 parser nodes" file=/usr/local/etc/crowdsec/parsers/s01-parse/sshd-logs.yaml stage=s01-parse
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 1 parser nodes" file=/usr/local/etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml stage=s02-enrich
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 1 parser nodes" file=/usr/local/etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml stage=s02-enrich
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 8 nodes from 3 stages"
time="2024-04-28T22:23:34+02:00" level=info msg="No postoverflow parsers to load"
time="2024-04-28T22:23:34+02:00" level=info msg="Loading 4 scenario files"
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=hidden-darkness name=crowdsecurity/opnsense-gui-bf
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=divine-darkness name=crowdsecurity/ssh-slow-bf
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=billowing-cloud name=crowdsecurity/ssh-slow-bf_user-enum
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=icy-voice name=firewallservices/pf-scan-multi_ports
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=divine-flower name=crowdsecurity/ssh-bf
time="2024-04-28T22:23:34+02:00" level=info msg="Adding leaky bucket" cfg=spring-river name=crowdsecurity/ssh-bf_user-enum
time="2024-04-28T22:23:34+02:00" level=info msg="Loaded 6 scenarios"
time="2024-04-28T22:23:34+02:00" level=info msg="loading acquisition file : /usr/local/etc/crowdsec/acquis.yaml"
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern /var/log/nginx/*.log" type=file
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern ./tests/nginx/nginx.log" type=file
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern /var/log/auth.log" type=file
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern /var/log/syslog" type=file
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern /var/log/httpd-access.log" type=file
time="2024-04-28T22:23:34+02:00" level=warning msg="No matching files for pattern /var/log/httpd-error.log" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="loading acquisition file : /usr/local/etc/crowdsec/acquis.d/opnsense.yaml"
time="2024-04-28T22:23:34+02:00" level=info msg="Force add watch on /var/log/audit" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Adding file /var/log/audit/latest.log to datasources" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Force add watch on /var/log/lighttpd" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Adding file /var/log/lighttpd/latest.log to datasources" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Force add watch on /var/log/filter" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Adding file /var/log/filter/latest.log to datasources" type=file
time="2024-04-28T22:23:34+02:00" level=info msg="Starting processing data"
time="2024-04-28T22:23:34+02:00" level=info msg="Error machine login for : ent: machine not found "
time="2024-04-28T22:23:34+02:00" level=info msg="retrying in 0 seconds (attempt 2 of 2)"
time="2024-04-28T22:23:34+02:00" level=info msg="Error machine login for : ent: machine not found "
time="2024-04-28T22:23:34+02:00" level=fatal msg="starting outputs error : authenticate watcher (): API error: ent: machine not found"
Hopefully going to be fixed with CrowdSec 1.6.1 - ETA "soon" - where they reworked most of the service management.