Elasticsearch (Usage)

If you want to know how to setup Elasticsearch on your local environment, please check the setup guide in VMS.

All usage of Elasticsearch is contained in the VaasCore application. We're using the elasticsearch package to access Elasticsearch from Elixir.

It's good to know that in VaasCore we only use Elasticsearch to retrieve documents from the index. The indexing itself and the definitions of documents and their mappings are all made in the VMS project.

Configuration

All configuration for Elasticsearch can be found in vaas_core/config/config.exs and vaas_core/config/test.exs. The former should be extended for each new Schema that is added to the index, the latter should not require any modifications.

Boilerplate

A little bit of boilerplate is required to set up Elasticsearch in Elixir. These files can be found in vaas_core/lib/vaas_core/elasticsearch:

  • cluster.ex - defines the Elasticsearch cluster to use
  • document.ex - defines the implementations for the Elasticsearch.Document protocol, for each Schema that has a document representation in Elasticsearch. Normally you would define the entire document and mappings here, but we do this in VMS
  • mock.ex - mock for Elasticsearch API to be used in tests
  • store.ex - boilerplate to connect Elasticsearch and Ecto

Querying Elasticsearch from Elixir

A full explanation of how Elasticsearch queries work can be found in Elasticsearch (usage) in VMS

There are two parts, both responsible for different parts of the querying flow:

  1. The Query module VaasCore.Elasticsearch.Query
  2. The Search context VaasCore.Search

Query Module

This module contains all building blocks to build a query that can be send to Elasticsearch. It's a little bit more limited than the QueryBuilder in VMS, as (for now) less functionality is required. If you want to add more / change functionality in the way we retrieve documents from Elasticsearch, this is the place to make your changes.

The module should be pretty self-explanatory. The public functions take a query string like "dogs" and some options, and return a query as a Map that can be fed to Elasticsearch, which is exactly what we do in the Search context.

If you want to learn more about the format of Elasticsearch queries, please check the documentation for the Query DSL

Search Context

This context contains a few functions that take a query from Elasticsearch.Query and return either a response from Elasticsearch (search/2) or an array of matching documents, as Structs retrieved from the database (productions/1 and shows/1).

These are the methods you will use from a controller. The normal workflow is something like this:

{:ok, query} = Query.productions_query("dogs", filters, limit, offset)
{:ok, productions, total_found} = Search.productions(query)

Using Elasticsearch in tests

You don't need to add anything special to your tests. As soon as your tests hits code calling VaasCore.Elasticsearch.Cluster, it will use the VaasCore.Elasticsearch.Mock to return the appropriate (mocked) response.

The only thing you need to do is add a new implementation of request/5 in VaasCore.Elasticsearch.Mock, in case your tests executes a request that hasn't been covered by the existing mocks.

Last Updated: 3/28/2019, 3:21:10 PM