[SOLR-12615] HashQParserPlugin will throw an NPE for string hash key and documents have empty value - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 7.5, 8.0
Component/s: None
Labels:
None

Description

If I index documents where the partitionKeys keys field is missing from a few docs then the stream expression throws an NPE

docs:

[
{"id" : "1", "search_term_s" : "query1"},
{"id" : "2", "search_term_s" : "query1"},
{"id" : "3"},
{"id" : "4"}
]

query

search(test_empty, q="*:*", fl="search_term_s,id" , sort="search_term_s desc", qt="/export", partitionKeys="search_term_s")

logs

INFO - 2018-08-03 01:44:36.156; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/stream params={expr=search(test_empty,+q%3D"*:*",+fl%3D"search_term_s,id"+,+sort%3D"search_term_s+desc",+qt%3D"/export",+partitionKeys%3D"search_term_s")&_=1533260573672} status=0 QTime=11
INFO - 2018-08-03 01:44:36.160; [ ] org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
INFO - 2018-08-03 01:44:36.162; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.common.cloud.ZkStateReader; Updated live nodes from ZooKeeper... (0) -> (1)
INFO - 2018-08-03 01:44:36.164; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at localhost:9888 ready
INFO - 2018-08-03 01:44:36.207; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/export params={q=*:*&distrib=false&indent=off&fl=search_term_s,id&sort=search_term_s+desc&partitionKeys=search_term_s&fq={!hash+workers%3D1+worker%3D0}&wt=json&version=2.2} status=500 QTime=36
ERROR - 2018-08-03 01:44:36.209; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.servlet.HttpSolrCall; null:java.io.IOException: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:743)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:463)
at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:151)
at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:140)
at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1196)
at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:836)
at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1044)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1563)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1439)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586)
at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:375)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
at org.apache.solr.handler.ExportHandler.handleRequestBody(ExportHandler.java:37)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
... 54 more
Caused by: java.lang.NullPointerException
at org.apache.lucene.util.CharsRefBuilder.copyUTF8Bytes(CharsRefBuilder.java:122)
at org.apache.solr.schema.FieldType.indexedToReadable(FieldType.java:378)
at org.apache.solr.search.HashQParserPlugin$BytesHash.hashCode(HashQParserPlugin.java:300)
at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
... 55 more

ERROR - 2018-08-03 01:44:36.213; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.io.stream.ExceptionStream; java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397)
at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83)
at org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539)
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181)
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209)
at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325)
at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120)
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394)
... 49 more
Caused by: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:222)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more

The problem lies with HashQParserPlugin where we try to partition the data and if a document doesn't have a value for the partition field it throws an NPE

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

SOLR-12615.patch
04/Aug/18 17:36
4 kB
Varun Thacker
SOLR-12615.patch
03/Aug/18 22:15
3 kB
Varun Thacker

Activity

People

Assignee:: Unassigned

Reporter:: Varun Thacker

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 03/Aug/18 01:50

Updated:: 21/Nov/19 00:42

Resolved:: 05/Aug/18 18:28