1
Zenarmor (Sensei) / Zenarmor External Elastic Database (Permission and TLS Issue)
« on: March 04, 2024, 03:21:36 pm »
New to OPNSense and trying out Zenarmor. Been trying the whole day and trying to set up external elastic-search.
What I want to achieve:
- Self-Signed cert to work
- Create user with appropriate permission for zenarmor to use:
Steps I have taken:
- Copy the generated ca cert and key and import into OPNSense "SYSTEM: TRUST: CERTIFICATES"
- Setup Zenarmour and Proceed as Usual...
What Happened:
- After copying the Cert into OPNSense, at least I am getting "Reporting Database" is running
- But when I go into Reporting & Data > Reporting Database Settings > "Perform Index Check" I get "Remote database connection failed" However this does not show up when I reinstall zenamour and add it at the installation page.
- Indicies still get created though
- So I am not very sure whether it is a cert issue or a permission issue
- Creating a non self-signed, no user account elastic does not have this issue.
Permission Given for the Zenamor Elastic User:
Cluster Privileges: manage_index_templates, manage_ilm, monitor
Index Privileges: All where given [all, write, delete, manage, manage_ilm, create_index, auto_configure]
Even though CA cert were installed already, any idea what could have went wrong?
Docker Compose
What I want to achieve:
- Self-Signed cert to work
- Create user with appropriate permission for zenarmor to use:
Steps I have taken:
- Copy the generated ca cert and key and import into OPNSense "SYSTEM: TRUST: CERTIFICATES"
- Setup Zenarmour and Proceed as Usual...
What Happened:
- After copying the Cert into OPNSense, at least I am getting "Reporting Database" is running
- But when I go into Reporting & Data > Reporting Database Settings > "Perform Index Check" I get "Remote database connection failed" However this does not show up when I reinstall zenamour and add it at the installation page.
- Indicies still get created though
- So I am not very sure whether it is a cert issue or a permission issue
- Creating a non self-signed, no user account elastic does not have this issue.
Permission Given for the Zenamor Elastic User:
Cluster Privileges: manage_index_templates, manage_ilm, monitor
Index Privileges: All where given [all, write, delete, manage, manage_ilm, create_index, auto_configure]
- alert\*
- conn\*
- dns\*
- http*
- sip*
- tls*
Code: [Select]
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Even though CA cert were installed already, any idea what could have went wrong?
Docker Compose
Code: [Select]
version: "3.8"
volumes:
certs:
es01-data:
kibana01-data:
networks:
elk-network:
services:
# Only for initial setup
setup:
container_name: setup
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
user: "0"
command: >
bash -c '
if [ x${ELASTIC_PASSWORD} == x ]; then
echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
exit 1;
elif [ x${KIBANA_PASSWORD} == x ]; then
echo "Set the KIBANA_PASSWORD environment variable in the .env file";
exit 1;
fi;
if [ ! -f config/certs/ca.zip ]; then
echo "Creating CA";
bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
unzip config/certs/ca.zip -d config/certs;
fi;
if [ ! -f config/certs/certs.zip ]; then
echo "Creating certs";
echo -ne \
"instances:\n"\
" - name: elk01-es01\n"\
" dns:\n"\
" - elk01-es01\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
" - 10.0.0.27\n"\
" - name: elk01-kibana01\n"\
" dns:\n"\
" - elk01-kibana01\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
" - 10.0.0.27\n"\
> config/certs/instances.yml;
bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
unzip config/certs/certs.zip -d config/certs;
fi;
echo "Setting file permissions"
chown -R root:root config/certs;
find . -type d -exec chmod 750 \{\} \;;
find . -type f -exec chmod 640 \{\} \;;
echo "Waiting for Elasticsearch availability";
until curl -s --cacert config/certs/ca/ca.crt https://elk01-es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
echo "Setting kibana_system password";
until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://elk01-es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
echo "All done!";
'
healthcheck:
test: ["CMD-SHELL", "[ -f config/certs/elk01-es01/elk01-es01.crt ]"]
interval: 1s
timeout: 5s
retries: 120
networks:
- elk-network
env_file:
- stack.env
elk01-es01:
depends_on:
setup:
condition: service_healthy
container_name: elk01-es01
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
labels:
co.elastic.logs/module: elasticsearch
volumes:
- certs:/usr/share/elasticsearch/config/certs
- es01-data:/usr/share/elasticsearch/data
ports:
- ${ES_PORT}:9200
environment:
- node.name=elk01-es01
- cluster.name=${CLUSTER_NAME}
- discovery.type=single-node
- network.host=0.0.0.0
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/elk01-es01/elk01-es01.key
- xpack.security.http.ssl.certificate=certs/elk01-es01/elk01-es01.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/elk01-es01/elk01-es01.key
- xpack.security.transport.ssl.certificate=certs/elk01-es01/elk01-es01.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
- elk-network
env_file:
- stack.env
elk01-kibana01:
depends_on:
elk01-es01:
condition: service_healthy
container_name: elk01-kibana
image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
labels:
co.elastic.logs/module: kibana
volumes:
- certs:/usr/share/kibana/config/certs
- kibana01-data:/usr/share/kibana/data
ports:
- ${KIBANA_PORT}:5601
environment:
- SERVERNAME=kibana
- ELASTICSEARCH_HOSTS=https://elk01-es01:9200
- ELASTICSEARCH_USERNAME=kibana_system
- ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
- ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
- XPACK_SECURITY_ENCRYPTIONKEY=${ENCRYPTION_KEY}
- XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${ENCRYPTION_KEY}
- XPACK_REPORTING_ENCRYPTIONKEY=${ENCRYPTION_KEY}
healthcheck:
test:
[
"CMD-SHELL",
"curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
- elk-network
env_file:
- stack.env