Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22973

Handle 0 length batches in LlapArrowRowRecordReader

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: llap

      Description

      In https://issues.apache.org/jira/browse/HIVE-22856, we allowed LlapArrowBatchRecordReader to permit 0 length arrow batches. LlapArrowRowRecordReader which is a wrapper over LlapArrowBatchRecordReader should also handle this.

      On one of the systems (cannot be reproduced easily) where we were running test TestJdbcWithMiniLlapVectorArrow, we saw following exception -

      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 30.173 s <<< FAILURE! - in org.apache.hive.jdbc.TestJdbcWithMiniLlapVectorArrow
      testLlapInputFormatEndToEnd(org.apache.hive.jdbc.TestJdbcWithMiniLlapVectorArrow)  Time elapsed: 6.476 s  <<< ERROR!
      java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0
      	at org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:80)
      	at org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:41)
      	at org.apache.hive.jdbc.BaseJdbcWithMiniLlap.processQuery(BaseJdbcWithMiniLlap.java:540)
      	at org.apache.hive.jdbc.BaseJdbcWithMiniLlap.processQuery(BaseJdbcWithMiniLlap.java:504)
      	at org.apache.hive.jdbc.BaseJdbcWithMiniLlap.testLlapInputFormatEndToEnd(BaseJdbcWithMiniLlap.java:236)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
      	at org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:77)
      	... 13 more
      

      cc mahesh kumar behera Jason Dere

        Attachments

        1. HIVE-22973.01.patch
          5 kB
          Shubham Chaurasia
        2. HIVE-22973.02.patch
          5 kB
          Jason Dere

          Issue Links

            Activity

              People

              • Assignee:
                ShubhamChaurasia Shubham Chaurasia
                Reporter:
                ShubhamChaurasia Shubham Chaurasia
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h