[LUCENE-10509] Performance degraded after upgrade from 8.8.2 to 8.9.0

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 8.8.2
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Rajesh T
• Created: 08/Apr/22 10:39
• Updated: 28/Aug/22 16:38
• Resolved: 08/Apr/22 13:18

Description

We are planning to upgrade from elasticsearch 7.7.1 to opensearch 1.1.0 (with lucene version 8.9.0). We have noticed that the performance of opensearch 1.1.0 is worse than elasticsearch 7.7.1 due to lucene 8.9.0 version. Whereas performance of opensearch 1.0.1 (with lucene version 8.8.2) is almost same as elasticsearch 7.7.1 version.

We have tested following scenarios and observed the slowness is caused by lucene 8.9.0 version. The performance is degraded by 50% for cardinality aggregations.

Elasticsearch 7.7.1 version (with lucene-core 8.5.1) : Fast

OpenSearch 1.0.1 version (with lucene-core 8.8.2) : Fast

OpenSearch 1.1.0 version (with lucene-core 8.9.0) : Slow

OpenSearch 1.1.0 version (with lucene-core 8.8.2) : Fast

This is the snippet of openseach code which is running slow with lucene 8.9.0

QueryBuilder qb = QueryBuilders.boolQuery().mustNot(QueryBuilders.termQuery("__id.keyword", randomValue));
CardinalityAggregationBuilder agg = AggregationBuilders
        .cardinality("somename")
        .field("__id.keyword");
return client.prepareSearch(index).setQuery(qb).addAggregation(agg);
 

Please let us know if this is something that can be fixed.