[HBASE-18879] HBase FilterList cause KeyOnlyFilter not work

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.4
• Component/s: Filters

Environment

OS： Red Hat 4.4.7-11
Hadoop: 2.6.4
Hbase: 1.2.4

• Hadoop Flags: Reviewed
• Assignee: Zheng Hu
• Reporter: ZHA_Moonlight
• Created: 26/Sep/17 03:52
• Updated: 25/Oct/17 16:57
• Resolved: 16/Oct/17 03:30

Description

when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter before FilterList, the KeyOnlyFilter may not work, means it will also grab the cell values:

List<Filter> filters = new ArrayList<Filter>();
        Filter filter1 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
                CompareOp.EQUAL, Bytes.toBytes("value1"));
        Filter filter2 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
                CompareOp.EQUAL, Bytes.toBytes("value2"));
        filters.add(filter1);
        filters.add(filter2);

        FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL, 
                new KeyOnlyFilter(),
                new FilterList(Operator.MUST_PASS_ONE, filters));

use the above code as filter to scan a table, it will return the cells with value instead of only return the key, if we put KeyOnlyFilter after FilterList as following, it works well.

FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL,
                new FilterList(Operator.MUST_PASS_ONE, filters),
                new KeyOnlyFilter());

the cause should due to the following code at hbase-client FilterList.java

@Override
  @edu.umd.cs.findbugs.annotations.SuppressWarnings(value="SF_SWITCH_FALLTHROUGH",
    justification="Intentional")
  public ReturnCode filterKeyValue(Cell v) throws IOException {
    this.referenceKV = v;

    // Accumulates successive transformation of every filter that includes the Cell:
    Cell transformed = v;

    ReturnCode rc = operator == Operator.MUST_PASS_ONE?
        ReturnCode.SKIP: ReturnCode.INCLUDE;
    int listize = filters.size();
    for (int i = 0; i < listize; i++) {
      Filter filter = filters.get(i);
      if (operator == Operator.MUST_PASS_ALL) {
        if (filter.filterAllRemaining()) {
          return ReturnCode.NEXT_ROW;
        }
LINE1      ReturnCode code = filter.filterKeyValue(v);{color}        
switch (code) {
        // Override INCLUDE and continue to evaluate.
        case INCLUDE_AND_NEXT_COL:
          rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
        case INCLUDE:
LINE2          transformed = filter.transformCell(transformed);{color}          
continue;
        case SEEK_NEXT_USING_HINT:
          seekHintFilter = filter;
          return code;
        default:
          return code;
        }
      }

notice the “LINE1”,"LINE2" , first line is a recursive invocation, it will assign a Cell results to the FilterList.transformedKV(we call it A), the results is from the FilterList with 2 SingleColumnValueFilter, so A with contains the cell value, while the second line with return A to the var transformed.
back to the following loop, we can see the FilterList return results is var "transformed " which will override in each loop, so the value is determined by the last filter, so the order of KeyOnlyFilter will impact the results.

 Cell transformed = v;

    ReturnCode rc = operator == Operator.MUST_PASS_ONE?
        ReturnCode.SKIP: ReturnCode.INCLUDE;
    int listize = filters.size();
    for (int i = 0; i < listize; i++) {
      Filter filter = filters.get(i);
      if (operator == Operator.MUST_PASS_ALL) {
        if (filter.filterAllRemaining()) {
          return ReturnCode.NEXT_ROW;
        }
        ReturnCode code = filter.filterKeyValue(v);
        switch (code) {
        // Override INCLUDE and continue to evaluate.
        case INCLUDE_AND_NEXT_COL:
          rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
        case INCLUDE:
          transformed = filter.transformCell(transformed);
          continue;
        case SEEK_NEXT_USING_HINT:
          seekHintFilter = filter;
          return code;
        default:
          return code;
        }
      

Issue Links

• relates to: HBASE-18410 FilterList Improvement.
• links to: Review Board (HBASE-18410)