[DRILL-4020] The not-equal operator returns incorrect results when used on the HBase row key

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.0, 1.3.0, 1.4.0, 1.5.0
• Fix Version/s: 1.14.0
• Component/s: Storage - HBase
• Labels: ready-to-commit

Environment

Drill Sandbox

• Assignee: Akihiko Kusanagi
• Reporter: Akihiko Kusanagi
• Reviewer: Parth Chandra
• Created: 03/Nov/15 18:11
• Updated: 02/Jul/18 06:30
• Resolved: 02/Jul/18 06:30

Description

Create a test HBase table:

hbase> create 'table', 'f'
hbase> put 'table', 'row1', 'f:c', 'value1'
hbase> put 'table', 'row2', 'f:c', 'value2'
hbase> put 'table', 'row3', 'f:c', 'value3'

The table looks like this:

0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table`;
+---------+
| EXPR$0  |
+---------+
| row1    |
| row2    |
| row3    |
+---------+
1 row selected (4.596 seconds)

However, this query returns incorrect results when a not-equal operator is used on the row key:

0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table` WHERE row_key <> 'row1';
+---------+
| EXPR$0  |
+---------+
| row1    |
| row2    |
| row3    |
+---------+
1 row selected (0.573 seconds)

In the query plan, there is no RowFilter:

00-00    Screen
00-01      Project(EXPR$0=[CONVERT_FROMUTF8($0)])
00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])

When the query has multiple not-equal operators, it works fine:

0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
+---------+
| EXPR$0  |
+---------+
| row3    |
+---------+
1 row selected (0.255 seconds)

In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:

00-00    Screen
00-01      Project(EXPR$0=[CONVERT_FROMUTF8($0)])
00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], columns=[`row_key`]]])

Issue Links

• links to: GitHub Pull Request #309