[SOLR-9246] Errors for Streaming Expressions using JDBC (Oracle) stream source

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.0.1
• Fix Version/s: 6.0.2, 6.1.1, 6.2, 7.0

Environment

Windows 7

• Assignee: Dennis Gove
• Reporter: Hui Liu
• Created: 23/Jun/16 16:17
• Updated: 31/Oct/16 13:46
• Resolved: 29/Jun/16 19:20

Description

I have Solr 6.0.0 installed on my PC (windows 7), I was experimenting with ‘Streaming Expression’ by using Oracle jdbc as the
stream source, but got 'null pointer' errors, below is the details on how to reproduce this error:

1. create a collection 'document6' which only contain long and string data type,

schema.xml for Solr collection 'document6': (newly created empty collections with 2 shards)
===========================================================================================
<schema name="document6" version="1.1">
<types>
<fieldType name="uuid" class="solr.UUIDField" />
<fieldType name="bigint" class="solr.TrieLongField" />
<fieldType name="string" class="solr.StrField" sortMissingLast="true" docValues="true" />
<fieldType name="long" class="solr.TrieLongField" docValues="true" precisionStep="0" positionIncrementGap="0"/>
<fieldType name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
</analyzer>
</fieldType>
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
</types>
<fields>
<field name="version" type="long" indexed="true" stored="true" multiValued="false"/>
<field name="document_id" type="long" indexed="true" stored="true" docValues="true"/>
<field name="sender_msg_dest" type="string" indexed="true" stored="true" docValues="true"/>
<field name="recip_msg_dest" type="string" indexed="true" stored="true" docValues="true"/>
<field name="document_type" type="string" indexed="true" stored="true" docValues="true"/>
<field name="document_key" type="string" indexed="true" stored="true" docValues="true"/>
</fields>
<defaultSearchField>document_id</defaultSearchField>
<uniqueKey>document_id</uniqueKey>
</schema>

2. create a new Oracle (version 11.2.0.3) table 'document6' that only contain columns whose jdbc type is long and string,

create table document6
(document_id number(12) not null,
sender_msg_dest varchar2(256),
recip_msg_dest varchar2(256),
document_type varchar2(20),
document_key varchar2(100));

loaded 9 records;

Oracle table 'document6': (newly created Oracle table with 9 records)
=====================================================================
QA_DOCREP@qlgdb1 > desc document6
Name Null? Type
----------------------------------------- -------- ----------------------------
DOCUMENT_ID NOT NULL NUMBER(12)
SENDER_MSG_DEST VARCHAR2(256)
RECIP_MSG_DEST VARCHAR2(256)
DOCUMENT_TYPE VARCHAR2(20)
DOCUMENT_KEY VARCHAR2(100)

3. tried this jdbc streaming expression in my browser, getting the error stack (see below)

http://localhost:8988/solr/document6/stream?expr=jdbc(connection="jdbc:oracle:thin:qa_docrep/[e-mail]:1521/qlgdb",sql="SELECT document_id,sender_msg_dest,recip_msg_dest,document_type,document_key FROM document6",sort="document_id asc",driver="oracle.jdbc.driver.OracleDriver")

errors in solr.log
==================
2016-06-23 14:07:02.833 INFO (qtp1389647288-139) [c:document6 s:shard2 r:core_node1 x:document6_shard2_replica1] o.a.s.c.S.Request [document6_shard2_replica1] webapp=/solr path=/stream params=

{expr=jdbc(connection%3D"jdbc:oracle:thin:qa_docrep/[e-mail]:1521/qlgdb",sql%3D"SELECT+document_id,sender_msg_dest,recip_msg_dest,document_type,document_key+FROM+document6",sort%3D"document_id+asc",driver%3D"oracle.jdbc.driver.OracleDriver")}

status=0 QTime=1
2016-06-23 14:07:05.282 ERROR (qtp1389647288-139) [c:document6 s:shard2 r:core_node1 x:document6_shard2_replica1] o.a.s.c.s.i.s.ExceptionStream java.lang.NullPointerException
at org.apache.solr.client.solrj.io.stream.JDBCStream.read(JDBCStream.java:305)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.read(ExceptionStream.java:64)
at org.apache.solr.handler.StreamHandler$TimerStream.read(StreamHandler.java:374)
at org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:305)
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167)
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183)
at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299)
at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95)
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)

2016-06-23 14:07:05.285 ERROR (qtp1389647288-139) [c:document6 s:shard2 r:core_node1 x:document6_shard2_replica1] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: java.lang.AbstractMethodError: Method oracle/jdbc/driver/T4CStatement.isClosed()Z is abstract
at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:605)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:475)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AbstractMethodError: Method oracle/jdbc/driver/T4CStatement.isClosed()Z is abstract
at oracle.jdbc.driver.T4CStatement.isClosed(T4CStatement.java)
at org.apache.solr.client.solrj.io.stream.JDBCStream.close(JDBCStream.java:287)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.close(ExceptionStream.java:79)
at org.apache.solr.handler.StreamHandler$TimerStream.close(StreamHandler.java:357)
at org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:317)
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167)
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183)
at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299)
at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95)
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
... 26 more

Here is the feedback from Joel Bernstein:
=======================================
I think we're going to have to add some debugging into the code to find what's going on. On line 225 in JDBCStream it's getting the class name for each column. It would be good know what the class names are that the Oracles driver is returning.

https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.0.0/solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/JDBCStream.java

We probably need to throw an exception that includes the class name to help users report what different drivers using for the classes.