Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3954

HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage - HBase
    • Labels:
      None

      Description

      Currently, the HBase tests (e.g., TestHBaseFilterPushDown) use only one region.

      That causes them to miss detecting a bug in creating and/or resolving dummy fields (NullableIntVectors for referenced but non-existent fields) somewhere between reading from HBase and wherever dummy fields are supposed to be get resolved (or maybe two separate bugs).

      Reproduction:

      In HBaseTestsSuite, change the line:

                  UTIL.startMiniHBaseCluster(1, 1);
      

      to:

                  UTIL.startMiniHBaseCluster(1, 3);
      

      and change the line:

          TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
      

      to:

          TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
      

      .

      Run unit test class TestHBaseFilterPushDown.

      Depending on which region gets processed first (it's non-deteministic), test methods testFilterPushDownOrRowKeyEqualRangePred and
      testFilterPushDownMultiColumns get exceptions like this:

      java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
      	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
      	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
      	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
      	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
      	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
      	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
      	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
      	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
      	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
      	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
      	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
      	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
      	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
      	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      

      See DRILL-3955.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dsbos Daniel Barclay
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: