Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10475

DIH JdbcDataSource SqlEntityProcessor - Support Child-free parents after all children has been consumed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 5.5.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Solr on Linux machine
      Oracle Data base (11g)

      Description

      Assume DIH's feature of zipping parents and children is used. Here is sample dih-config.xml:

          <dataConfig>
      	<document>
      		<entity name="parent" processor="SqlEntityProcessor" query="SELECT * FROM PARENT ORDER BY id">		
      			<entity name="child_1" processor="SqlEntityProcessor"
      				where="parent_id=parent.id" query="SELECT * FROM CHILD_1 ORDER BY parent_id" join="zipper" >
      			</entity>			
      		</entity>
      	</document>
          </dataConfig>
      

      One might come up with the issue when:

      • Oracle Database is used;
      • at some point of joining there are no more children documents; thus the child's ResultSet is closed;
      • parent documents are not finished; thus ResultSet is still in process;
      • attempting to find child for the next parent is failing due to closed ResultSet

      Here is stacktrace:

      org.apache.solr.handler.dataimport.DataImportHandlerException: java.sql.SQLRecoverableException: Closed Resultset: next
      at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:61)
      at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:434)
      at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:350)
      at com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1216)
      at org.apache.solr.handler.dataimport.Zipper.supplyNextChild(Zipper.java:65)
      at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:127)
      at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
      at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
      at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
      at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
      at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
      at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
      at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
      at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
      at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
      at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
      Caused by: java.sql.SQLRecoverableException: Closed Resultset: next
      at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:238)
      at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:426)
      ... 14 more
      

        Attachments

        1. SOLR-10475.patch
          2 kB
          Andrey Kudryavtsev

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              pvasilyev Pavel Vasilyev
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: