Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18410 FilterList Improvement.
  3. HBASE-18879

HBase FilterList cause KeyOnlyFilter not work

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.2.4
    • None
    • Filters
    • None
    • OS: Red Hat 4.4.7-11
      Hadoop: 2.6.4
      Hbase: 1.2.4

    • Reviewed

    Description

      when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter before FilterList, the KeyOnlyFilter may not work, means it will also grab the cell values:

      List<Filter> filters = new ArrayList<Filter>();
              Filter filter1 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
                      CompareOp.EQUAL, Bytes.toBytes("value1"));
              Filter filter2 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"),
                      CompareOp.EQUAL, Bytes.toBytes("value2"));
              filters.add(filter1);
              filters.add(filter2);
      
              FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL, 
                      new KeyOnlyFilter(),
                      new FilterList(Operator.MUST_PASS_ONE, filters));
      

      use the above code as filter to scan a table, it will return the cells with value instead of only return the key, if we put KeyOnlyFilter after FilterList as following, it works well.

      FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL,
                      new FilterList(Operator.MUST_PASS_ONE, filters),
                      new KeyOnlyFilter());
      

      the cause should due to the following code at hbase-client FilterList.java

      @Override
        @edu.umd.cs.findbugs.annotations.SuppressWarnings(value="SF_SWITCH_FALLTHROUGH",
          justification="Intentional")
        public ReturnCode filterKeyValue(Cell v) throws IOException {
          this.referenceKV = v;
      
          // Accumulates successive transformation of every filter that includes the Cell:
          Cell transformed = v;
      
          ReturnCode rc = operator == Operator.MUST_PASS_ONE?
              ReturnCode.SKIP: ReturnCode.INCLUDE;
          int listize = filters.size();
          for (int i = 0; i < listize; i++) {
            Filter filter = filters.get(i);
            if (operator == Operator.MUST_PASS_ALL) {
              if (filter.filterAllRemaining()) {
                return ReturnCode.NEXT_ROW;
              }
      LINE1      ReturnCode code = filter.filterKeyValue(v);{color}        
      switch (code) {
              // Override INCLUDE and continue to evaluate.
              case INCLUDE_AND_NEXT_COL:
                rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
              case INCLUDE:
      LINE2          transformed = filter.transformCell(transformed);{color}          
      continue;
              case SEEK_NEXT_USING_HINT:
                seekHintFilter = filter;
                return code;
              default:
                return code;
              }
            }
      

      notice the “LINE1”,"LINE2" , first line is a recursive invocation, it will assign a Cell results to the FilterList.transformedKV(we call it A), the results is from the FilterList with 2 SingleColumnValueFilter, so A with contains the cell value, while the second line with return A to the var transformed.
      back to the following loop, we can see the FilterList return results is var "transformed " which will override in each loop, so the value is determined by the last filter, so the order of KeyOnlyFilter will impact the results.

       Cell transformed = v;
      
          ReturnCode rc = operator == Operator.MUST_PASS_ONE?
              ReturnCode.SKIP: ReturnCode.INCLUDE;
          int listize = filters.size();
          for (int i = 0; i < listize; i++) {
            Filter filter = filters.get(i);
            if (operator == Operator.MUST_PASS_ALL) {
              if (filter.filterAllRemaining()) {
                return ReturnCode.NEXT_ROW;
              }
              ReturnCode code = filter.filterKeyValue(v);
              switch (code) {
              // Override INCLUDE and continue to evaluate.
              case INCLUDE_AND_NEXT_COL:
                rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH
              case INCLUDE:
                transformed = filter.transformCell(transformed);
                continue;
              case SEEK_NEXT_USING_HINT:
                seekHintFilter = filter;
                return code;
              default:
                return code;
              }
            
      

      Attachments

        1. HBASE-18879-HBASE-18410.v1.patch
          8 kB
          Zheng Hu
        2. HBASE-18879-HBASE-18410.v2.patch
          9 kB
          Zheng Hu

        Issue Links

          Activity

            People

              openinx Zheng Hu
              pengxu ZHA_Moonlight
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: