Elasticsearch 8x latest and greatest

Elasticsearch 8.0 became generaly available at the beginning of the year. The cornerstone of the 8x releases have been a number of performance, stability and security improvements. Apart from that new capabilities especially in the area of machine learning and NLP have also been introduced. In this article we will categorize some of the highlights of the Elasticsearch 8x release.

Security

In 7x and earlier versions Elasticsearch didn’t enable security features by default during installation. This however changed in 8x:

• activation of Elasticsearch is enabled through Kibana by means of an enrollmnent token, the same mechanism can be used to add new nodes to a cluster;
• authentication and authorization are enabled by default;
• TLS between cluster nodes and on the HTTP API are enabled by default;
• system indices have better protection by introducing a new allow_restricted_indices role.

Performance

Performance improvements are also an essential part for every system that can operate on large ammounts of data as Elasticsearch does. In 8x:

• faster indexing of certain field types such as range;
• additional storage optimizations have been implemented for certail field type such as text and keyword;
• performance improvements on ingest pipeline processing;
• faster execution of filters, range and date_histogram aggregations;
• faster execution of SQL queries by avoiding calculation of total number of hits using track_total_hits counter.

Stability

Stability is improvement by a number of capabilities such as:

• 7.x REST API compatibility;
• complete removal of mapping types deprecated in 7x;
• possibility to import and query indices created in Elasticsearch 5x and 6x;
• upgrade to latest Lucene 9.

New capabilities

New capabilities are introduced with a focus on machine learning and NLP:

• new k-nearest neighbor (kNN) search API;
• import of PyTorch NLP models for ingest processing;
• new frequent items aggregation;
• new random sampler aggregation (technical preview).

Apart from that a new Elasticearch Java API Client is introduced replacing the now deprecated High Level REST client.

Author: Martin Toshev