At the beginning there was only Elasticsearch …
It all started when Elasticsearch and Kibana license was changed from Apache 2.0 to EL (Elastic License) and SSPL (Server Side Public License). OpenSearch (formerly OpenDistro for Elasticsearch) is a fork of the Apache 2.0 licensed versions (7.10.2) of Elasticsearch and Kibana developed by AWS (Amazon Web Services).
The OpenSearch project
In a nutshell OpenSearch projects aims to be a completely open-source equivalent of Elasticsearch oferring many of the enterprise capabilities of Elasticsearch also for free. OpenSearch is developed by Amazon and offered as a managed AWS service, a replacement of the Elasticsearch one. The projects has also forked many of the other appliations in the Elastic stack like Kibana (renamed to OpenSearch Dashboards) and also the various Elasticsearch language clients. In essence OpenSearch and all applications around it evolve as separate products that have no compatility with newer versions of Elasticsearch. The capabilities of features in OpenSearch are developed as plugins (available in the public repositories of the OpenSearch Project organization in GitHub) including the ones that are distributed as part of the core Elasticsearch application. A new mechanism called extensions is being developed to serve as a replacement of the standard plugin system in OpenSearch (inherited by Elasticsearch) with the aim to solve its main disadvantages:
- run in the same process as OpenSearch;
- require cluster restart on plugin installation/update;
- require updates across OpenSearch versions.
<dependency><groupId>org.opensearch.client</groupId><artifactId>opensearch-rest-high-level-client</artifactId><version>2.9.0</version></dependency><dependency><groupId>org.opensearch.client</groupId><artifactId>spring-data-opensearch</artifactId><version>1.2.0</version></dependency>
Extensions run as separate processes that interact with the OpenSearch cluster and use the same protocol as OpenSearch nodes for communcation. They may also invoke actions on other extensions that enables a mechanism for communication between the various extensions. Some of the plugins in the OpenSearch ecosystem are being reimplemented as extensions.
OpenSearch also ships and open source distribution of Logstash which however is currently planned to be discontinued in favor of standard Logstash oferring by Elastic. In terms of Logstash later versions of the Elasticsearch output plugin are not compatible with OpenSearch and for that reason OpenSearch provides also a third-party output plugin for LogStash. In addition a new capability called OpenSearch Integrations is being developed as part of OpenSearch Dashboards which provides the possiblity to collect and visualize data from external sources into OpenSearch. In certain cases OpenSearch Integrations can be used as an alternative to Logstash.
In terms of Spring Framework OpenSearch also provides a third-party Spring Data OpenSearch framework, an equivalent to the Spring Data Elasticsearch one.
OpenSearch vs Elasticsearch: a brief comparison guide
Let’s do a brief comparison between traditional Elasticsearch and OpenSearch using a few different criteria.
Licensing and subscriptions
Elasticsearch provides a more restrictire license as of 7.11 for service providers and subscription-based model for enterprise Elasticsearch features. On the other hand there is no subscription model in OpenSearch: it is a fully open source project with certain limitations on features bound to the AWS OpenSearch offering.
Support
Elasticsearch provides support options as per subscription plan while OpenSearch has only community support and discussion can be conducted in the community forum of OpenSearch.
Documentation
Elasticsearch has a rich documentation with some nice capabilities like running examples directly in Kibana or copying them as CURL commands. OpenSearch has less examples and is not structured so well.
Roadmap
Elasticsearch does not have a public roadmap exposed and provides overview of new capabilities throughout i.e. blogpost. OpenSearch roadmap on the other hand is publicly available.
Migrating from Elasticsearch to OpenSearch (and vice-versa)
In the aim to position itself as a competitor product OpenSearch even provides a migration guide from Elasticsearch OSS and even a tool that aids that process. While no official guide on Elasticsearch a reverse process can be applied to migrate from OpenSearch to Elasticsearch in practice.
Let’s see an example on how to migrate a simple Java application that uses both Elasticsearch Java client and Spring Data Elasticsearch. The process may be as simple as:
- assuming we use High Level Elasticsearch client and Spring Data Elasticsearch we can simply replace the dependencies for OpenSearch Java High Level Rest client and the one for Spring Data OpenSearch:
<dependency><groupId>org.opensearch.client</groupId><artifactId>opensearch-rest-high-level-client</artifactId><version>2.9.0</version></dependency><dependency><groupId>org.opensearch.client</groupId><artifactId>spring-data-opensearch</artifactId><version>1.2.0</version></dependency>
- update imports: boils down to just replacing org.elasticsearch to org.opensearch for the Java client and no change for Spring Data framework.
- if you use a secure version of Elasticsearch which is typically the case with later Elasticsearch versions and use username, password and fingerpring to connect to Elasticsearch such as provided by the following method:
private static RestClientBuilder createClientBuilder() {
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials("elastic", "ZldXf5gREgh2OaE5HiIw"));
String fingerprint = "<fingerprint>";
SSLContext sslContext = TransportUtils.sslContextFromCaFingerprint(fingerprint);
RestClientBuilder builder = RestClient.builder(new HttpHost("localhost", 9200, "https"))
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder
.setSSLContext(sslContext)
.setDefaultCredentialsProvider(credentialsProvider);
}
});
return builder;
}
If you start OpenSearch by default you get a generated CA certificate and you can generate a fingerprint out of the certificate using a tool like openssl as follows:
openssl x509 -in ca.pem -sha256 -fingerprint | grep SHA256 | sed ‘s/://g’
After that you can replace that fingerprint along with the username and password in the code above. The sslContextFromCaFingerprint from the TransportUtils class can be copied over and reused when working with OpenSearch as it is not present in the OpenSearch Java client dependency.
- For Spring Data OpenSearch it is sufficient to define a proper client configuration component for i.e. a RestHighLevelClient as per previous step and use the same setup as for Spring Data Elasticsearch. There should also be a configuration class that extends from the AbstractOpenSearchConfiguration class provided by the Spring Data OpenSearch framework
Conclusion
OpenSearch provides a good open source alternative of Elasticsearch with many of its enterprise features offered for free. However Elasticsearch remains backend by a solid foundation and continues to invest into new features and performance improvements while OpenSearch tried to gain contributions more from the community. While it might be a good choice for applications that really need the core capabilities of an enterprise-level production-ready search offering Elasticsearch has still more to offer in terms certain aspects such as support, documentation and enterprise features missing from OpenSearch. The two applications will continue diverging over time and evolving as separate products.
steinhauer.software