Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4020

The not-equal operator returns incorrect results when used on the HBase row key

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0, 1.4.0, 1.5.0
    • Fix Version/s: 1.14.0
    • Component/s: Storage - HBase
    • Labels:
    • Environment:

      Drill Sandbox

      Description

      Create a test HBase table:

      hbase> create 'table', 'f'
      hbase> put 'table', 'row1', 'f:c', 'value1'
      hbase> put 'table', 'row2', 'f:c', 'value2'
      hbase> put 'table', 'row3', 'f:c', 'value3'
      

      The table looks like this:

      0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table`;
      +---------+
      | EXPR$0  |
      +---------+
      | row1    |
      | row2    |
      | row3    |
      +---------+
      1 row selected (4.596 seconds)
      

      However, this query returns incorrect results when a not-equal operator is used on the row key:

      0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table` WHERE row_key <> 'row1';
      +---------+
      | EXPR$0  |
      +---------+
      | row1    |
      | row2    |
      | row3    |
      +---------+
      1 row selected (0.573 seconds)
      

      In the query plan, there is no RowFilter:

      00-00    Screen
      00-01      Project(EXPR$0=[CONVERT_FROMUTF8($0)])
      00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
      

      When the query has multiple not-equal operators, it works fine:

      0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
      +---------+
      | EXPR$0  |
      +---------+
      | row3    |
      +---------+
      1 row selected (0.255 seconds)
      

      In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:

      00-00    Screen
      00-01      Project(EXPR$0=[CONVERT_FROMUTF8($0)])
      00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], columns=[`row_key`]]])
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nagix Akihiko Kusanagi
                Reporter:
                nagix Akihiko Kusanagi
                Reviewer:
                Parth Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: