[SOLR-13414] SolrSchema - Avoid NPE if Luke returns field with no type defined - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 7.3, 7.7.1
Fix Version/s: 7.7.2, 8.1, 9.0
Component/s: Parallel SQL
Labels:
None

Description

Summary
If the underlying Lucene index has fields defined but no type, SolrSchema fails with NPE. The index most likely has issues and would be better to delete/recreate the index. This ticket adds a null check to prevent the NPE and won't break on a potentially invalid index.

Initial Description
When attempting to create a JDBC sql query against a large collection (400m + records) we get a null error.

After initial discussion in solr-user I have been asked to open this ticket - The exception thrown does not provide sufficient detail to understand the underlying problem. Its it thought to be an issue with the schema not initialising correctly.

Attached is the managed-schema after a downconfig.

Stack trace from email thread:

Solr Admin UI Logging

java.io.IOException: Failed to execute sqlQuery 'select id from document limit 10' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select id from document limit 10": null
        at org.apache.solr.client.solrj.io.stream.JDBCStream.open(JDBCStream.java:271)
        at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
        at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:394)
        at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:78)
        at org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
        at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
        at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:152)
        at org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:386)
        at org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:292)
        at org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
        at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
        at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
        at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:788)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:525)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.Server.handle(Server.java:502)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Error while executing SQL "select id from document limit 10": null
        at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
        at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
        at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
        at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
        at org.apache.solr.client.solrj.io.stream.JDBCStream.open(JDBCStream.java:269)
        ... 49 more
Caused by: java.lang.NullPointerException
        at org.apache.calcite.plan.volcano.VolcanoPlanner.validate(VolcanoPlanner.java:891)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:866)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
        at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1496)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
        at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1496)
        at org.apache.calcite.plan.volcano.VolcanoPlanner.setRoot(VolcanoPlanner.java:308)
        at org.apache.calcite.tools.Programs$5.run(Programs.java:309)
        at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
        at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:186)
        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:319)
        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:784)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:639)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:609)
        at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
        at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
        at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
        at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)

solr.log

Error while executing SQL "select id from data limit 10": null

at
org.apache.solr.client.solrj.io.stream.JDBCStream.open(JDBCStream.java:271)
at
org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
at
org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:394)
at
org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:78)
at
org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
at
org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:152)
at
org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:386)
at
org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:292)
at org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
at
org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
at
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:788)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:525)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Error while executing SQL "select id from
data limit 10": null
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.solr.client.solrj.io.stream.JDBCStream.open(JDBCStream.java:269)
... 48 more
Caused by: java.lang.NullPointerException
at
org.apache.solr.handler.sql.SolrSchema.getRelDataType(SolrSchema.java:103)
at org.apache.solr.handler.sql.SolrTable.getRowType(SolrTable.java:83)
at org.apache.calcite.sql.validate.EmptyScope.resolve_(EmptyScope.java:159)
at
org.apache.calcite.sql.validate.EmptyScope.resolveTable(EmptyScope.java:99)
at
org.apache.calcite.sql.validate.DelegatingScope.resolveTable(DelegatingScope.java:202)
at
org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl(IdentifierNamespace.java:104)
at
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:167)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:939)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:920)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2954)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2939)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3181)
at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:939)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:920)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:220)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:895)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:605)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:550)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:264)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:784)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:639)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:609)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
... 50 more

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

before_starting_solr.png
23/Apr/19 21:12
547 kB
David Barnett
command_prompt.png
23/Apr/19 15:19
50 kB
David Barnett
luke_out.xml
25/Apr/19 15:29
26 kB
David Barnett
managed-schema
19/Apr/19 11:42
56 kB
David Barnett
new_solr.log
23/Apr/19 21:22
172 kB
David Barnett
new_solr-8983-console.log
23/Apr/19 21:22
2 kB
David Barnett
solr.log
23/Apr/19 14:55
210 kB
David Barnett
SOLR-13414.patch
25/Apr/19 17:58
3 kB
Kevin Risden
SOLR-13414.patch
19/Apr/19 16:16
0.9 kB
Joel Bernstein
solr-8983-console.log
25/Apr/19 11:18
5 kB
David Barnett
solr-8983-console.log
23/Apr/19 15:56
2 kB
David Barnett
solr-core-7.8.0-SNAPSHOT.jar
19/Apr/19 16:17
5.27 MB
Joel Bernstein

Issue Links

links to

GitHub Pull Request #655

Activity

People

Assignee:: Kevin Risden

Reporter:: David Barnett

Votes:: 0 Vote for this issue

Watchers:: 8 Start watching this issue

Dates

Created:: 19/Apr/19 11:42

Updated:: 08/Oct/19 14:50

Resolved:: 26/Apr/19 14:31

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

0.5h