[LUCENE-9791] Monitor (aka Luwak) has concurrency issues related to BytesRefHash#find

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 9.0, 8.7, 8.8
• Fix Version/s: 9.0, 8.9
• Component/s: core/other
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Paweł Bugalski
• Created: 19/Feb/21 12:39
• Updated: 28/Aug/22 16:16
• Resolved: 12/Mar/21 16:32

Description

org.apache.lucene.monitor.Monitor can sometimes NOT match a document that should be matched by one of registered queries if match operations are run concurrently from multiple threads. 

This is because sometimes in a concurrent environment TermFilteredPresearcher might not select a query that could later on match one of documents being matched.

Internally TermFilteredPresearcher is using a term acceptor: an instance of org.apache.lucene.monitor.QueryIndex.QueryTermFilter. QueryTermFilter is correctly initialized under lock and its internal state (a map of org.apache.lucene.util.BytesRefHash instances) is correctly published. Later one when those instances are used concurrently a problem with org.apache.lucene.util.BytesRefHash#find is triggered since it is not thread safe.

org.apache.lucene.util.BytesRefHash#find internally is using a private org.apache.lucene.util.BytesRefHash#equals method, which is using an instance field scratch1 as a temporary buffer to compare its ByteRef parameter with contents of ByteBlockPool. This is not thread safe and can cause incorrect answers as well as ArrayOutOfBoundException. 

__

 

Issue Links

• links to: GitHub Pull Request #8
• links to: GitHub Pull Request #2429