External Elasticsearch 'not running'

[apiods]

Hi,

Need some help please with external Elasticsearch 

I've been playing around with using a remote Elasticsearch instance in Azure, on an Ubuntu VM. A while back I had this working okay, although not secured (it was connecting to the ES instance over http://azure_ip:9200).
Then something broke, which was fine as it made me look at securing it properly

So, it seemed that one way to secure this was to configure SSL on the Elasticsearch installation with Nginx reverse proxy - which I did, and that appeared to work:

Code: [Select]

❯ curl -u elastic:changeme -kL https://search.domain.co
{
  "name" : "server1.cloudapp.azure.com",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "some-random-string",
  "version" : {
    "number" : "7.9.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "some-random-string",
    "build_date" : "2020-09-23T00:45:33.626720Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
Then, I tried setting up Sensei with a fresh config and deleted the '/usr/local/sensei/etc/.configdone' file.

During the setup wizard, it complained about the Database URL as just:

Code: [Select]

https://search.domain.coSo, adding the URL as

Code: [Select]

https://search.domain.co:443 seemed to work and i could complete the setup.

But, I then click on Dashboard and get the error:

Code: [Select]

Elasticsearch service is not running!
In order to view reports, you need to start Elasticsearch service.
Checking the Sensei config and resetting the DB url, it now errors with:

Code: [Select]

Elastic Search Database (https.//search.domain.co.443) cannot be reached. Please check your network connectivity and make sure the remote database is up and running.
But, running the test curl cmd from the opnsense shell works okay.

Any ideas on this error.
Or what's the recommended way to setup a secure, external ES instance ?

Thanks 

[apiods]

Have played around with this a little more and have now enabled TLS Encryption and HTTPS with ElasticSearch, refreshed the Sensei config but same results - the wizard went through fine, but checking the DB URL connection, it errors with:

Code: [Select]

Elastic Search Database (https.//search.domain.co.9200) cannot be reached
I did generate a self-signed certificate, no password, etc.

Testing from the shell works okay:

Code: [Select]

curl --user elastic:elastic123 --insecure -X GET "https://search.domain.co:9200/?pretty"
{
  "name" : "server1.cloudapp.azure.com",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "some-random-string",
  "version" : {
    "number" : "7.9.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "some-random-string",
    "build_date" : "2020-09-23T00:45:33.626720Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },

[apiods]

Hey @mb. Any suggestions or help on this one?

I'd like to upgrade to the Premium version, but really need a working install before I can do that 
thanks.

[sy]

Hi @apiods,

Can you send a bug report by selecting all checkboxes? It is the upper right corner of Sensei GUI.

[apiods]

Can you send a bug report by selecting all checkboxes? It is the upper right corner of Sensei GUI.

Done. Thanks