Elasticsearch connector: Some examples

Elasticsearch is a distributed, RESTful search and analytics engine. Omniscope offerst a connector to query and retrieve documents from elasticserch and use documents in the Datamanager app.

Block options are mapped to the bool compound query in elasticsearch

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html

Each leaf query can be one of Match, Term, Range and Exists:

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html

In out examples, the we do have two indices people and people2. The documents loaded on these indices look like this:

        {

          "birthday": "1989-03-10",

          "firstname": "Stuart",

          "address": {

            "city": "Stollings",

            "street": "16 Kingsland Avenue",

            "latitude": 4.535723,

            "state": "Arizona",

            "longitude": 4.353591

          },

          "gender": "male",

          "phone": "+1 (810) 553-3006",

          "surname": "Leon",

          "company": "Supportal",

          "comment": "Stuart is 29 years old and lives in Arizona, has 3 children and a great hobby called idle",

          "isActive": true,

          "email": "[email protected]",

          "hobby": "idle"

        }

In the index name field it is possible to specify the index of interest.

It is also possible to query both indices by using a wildcard search:

Ot the strong people,people2 if just the two indices are of interest (and not all the indices matching the wildcard).

Some examples

Most of the examples below will use the Filter clause, but the same operations are possible with the other clauses, according to their semantics:

- All documents where address.street contains "Veronica Place"

- All documents where address.street contains "Veronica" or "Place" (or both)

- All documents where the company is exactly "Zolar"

- All active people

- All the people born after 15/07/1982

- All people born between 15/07/1982 and 15/07/1989

- All people born between 15/07/1982 and 15/07/1989 , that do work for "Uni"

- All people born between 15/07/1982 and 15/07/1989 , that do not work for "Uni"  

- All people where the email is present, and born before the year 20000




Endpoints

In terms of elasticsearch endpoints, Omniscope will need access to the mapping endpoint to retrieve the schema of the index.
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html
Then the query string (a json string) is built based on the block option. The query is then submitted to the search api endpoint
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

That will open a "search context" (similar to a cursor in a RDBMS). Further records for the same query are retrieved from the scroll api endpoint
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html