Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8930

Filter evaluates KVs outside requested columns

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed
    • Tags:
      filters

      Description

      1- Fill row with some columns
      2- Get row with some columns less than universe - Use filter to print kvs
      3- Filter prints not requested columns

      Filter (AllwaysNextColFilter) always return ReturnCode.INCLUDE_AND_NEXT_COL and prints KV's qualifier

      SUFFIX_0 = 0
      SUFFIX_1 = 1
      SUFFIX_4 = 4
      SUFFIX_6 = 6

      P= Persisted
      R= Requested
      E= Evaluated
      X= Returned

      5580 5581 5584 5586 5590 5591 5594 5596 5600 5601 5604 5606 ...
        P P     P P     P P   ...
        R R R   R R R         ...
        E E     E E     E     ...
        X X     X X          
      ExtraColumnTest.java
          @Test
          public void testFilter() throws Exception {
      
              Configuration config = HBaseConfiguration.create();
              config.set("hbase.zookeeper.quorum", "myZK");
      
              HTable hTable = new HTable(config, "testTable");
              byte[] cf = Bytes.toBytes("cf");
              byte[] row = Bytes.toBytes("row");
      
      
              byte[] col1 = new QualifierConverter().objectToByteArray(new Qualifier((short) 558, (byte) SUFFIX_1));
              byte[] col2 = new QualifierConverter().objectToByteArray(new Qualifier((short) 559, (byte) SUFFIX_1));
              byte[] col3 = new QualifierConverter().objectToByteArray(new Qualifier((short) 560, (byte) SUFFIX_1));
              byte[] col4 = new QualifierConverter().objectToByteArray(new Qualifier((short) 561, (byte) SUFFIX_1));
              byte[] col5 = new QualifierConverter().objectToByteArray(new Qualifier((short) 562, (byte) SUFFIX_1));
              byte[] col6 = new QualifierConverter().objectToByteArray(new Qualifier((short) 563, (byte) SUFFIX_1));
      
      
              byte[] col1g = new QualifierConverter().objectToByteArray(new Qualifier((short) 558, (byte) SUFFIX_6));
              byte[] col2g = new QualifierConverter().objectToByteArray(new Qualifier((short) 559, (byte) SUFFIX_6));
      
              byte[] col1v = new QualifierConverter().objectToByteArray(new Qualifier((short) 558, (byte) SUFFIX_4));
              byte[] col2v = new QualifierConverter().objectToByteArray(new Qualifier((short) 559, (byte) SUFFIX_4));
              byte[] col3v = new QualifierConverter().objectToByteArray(new Qualifier((short) 560, (byte) SUFFIX_4));
              byte[] col4v = new QualifierConverter().objectToByteArray(new Qualifier((short) 561, (byte) SUFFIX_4));
              byte[] col5v = new QualifierConverter().objectToByteArray(new Qualifier((short) 562, (byte) SUFFIX_4));
              byte[] col6v = new QualifierConverter().objectToByteArray(new Qualifier((short) 563, (byte) SUFFIX_4));
      
              // =========== INSERTION =============//
              Put put = new Put(row);
              put.add(cf, col1, Bytes.toBytes((short) 1));
              put.add(cf, col2, Bytes.toBytes((short) 1));
              put.add(cf, col3, Bytes.toBytes((short) 3));
              put.add(cf, col4, Bytes.toBytes((short) 3));
              put.add(cf, col5, Bytes.toBytes((short) 3));
              put.add(cf, col6, Bytes.toBytes((short) 3));
              hTable.put(put);
      
              put = new Put(row);
              put.add(cf, col1v, Bytes.toBytes((short) 10));
              put.add(cf, col2v, Bytes.toBytes((short) 10));
              put.add(cf, col3v, Bytes.toBytes((short) 10));
              put.add(cf, col4v, Bytes.toBytes((short) 10));
              put.add(cf, col5v, Bytes.toBytes((short) 10));
              put.add(cf, col6v, Bytes.toBytes((short) 10));
      
              hTable.put(put);
              hTable.flushCommits();
      
              //==============READING=================//
              Filter allwaysNextColFilter = new AllwaysNextColFilter();
      
              Get get = new Get(row);
              get.addColumn(cf, col1); //5581
              get.addColumn(cf, col1v); //5584
              get.addColumn(cf, col1g); //5586
              get.addColumn(cf, col2); //5591
              get.addColumn(cf, col2v); //5594        
              get.addColumn(cf, col2g); //5596
              
              get.setFilter(allwaysNextColFilter);
              get.setMaxVersions(1);
              System.out.println(get);
      
              Scan scan = new Scan(get);
      
              ResultScanner scanner = hTable.getScanner(scan);
              Iterator<Result> iterator = scanner.iterator();
      
              System.out.println("SCAN");
              while (iterator.hasNext()) {
                  Result next = iterator.next();
                  for (KeyValue kv : next.list()) {
                      System.out.println(new QualifierConverter().byteArrayToObject(kv.getQualifier()));
                  }
              }
          }
      
      }
      
      

      Requested 5581 5584 5586 5591 5594 5596
      NOT REQUESTED: 5561

      Sysout Filter

      \x00\x00\x1A\xBE\x00\x05^:\x00\x00\xA0X\x00\x00=\x1A/H0:\x02.\x01/1373577819267/Put/vlen=2/ts=2
      Qualifier{date=558, type=SUFFIX_1}
      \x00\x00\x1A\xBE\x00\x05^:\x00\x00\xA0X\x00\x00=\x1A/H0:\x02.\x02/1373577819272/Put/vlen=2/ts=3
      Qualifier{date=558, type=SUFFIX_4}
      \x00\x00\x1A\xBE\x00\x05^:\x00\x00\xA0X\x00\x00=\x1A/H0:\x02/\x01/1373577819267/Put/vlen=2/ts=2
      ualifier{date=559, type=SUFFIX_1}
      \x00\x00\x1A\xBE\x00\x05^:\x00\x00\xA0X\x00\x00=\x1A/H0:\x02/\x02/1373577819272/Put/vlen=2/ts=3
      Qualifier{date=559, type=SUFFIX_4}
       
      \x00\x00\x1A\xBE\x00\x05^:\x00\x00\xA0X\x00\x00=\x1A/H0:\x020\x01/1373577819267/Put/vlen=2/ts=2
      Qualifier{date=560, type=SUFFIX_1} (DATE 5601 NOT REQUESTED BUT EVALUATED)
      

      Sysout ExtraColumnTest

      {"timeRange":[0,9223372036854775807],"totalColumns":6,"cacheBlocks":true,"families":{"H0":["\\x02.\\x01","\\x02.\\x02","\\x02.\\x06","\\x02/\\x01"]},"maxVersions":1,"filter":"AllwaysNextColFilter","row":"\\x00\\x00\\x1A\\xBE\\x00\\x05^:\\x00\\x00\\xA0X\\x00\\x00=\\x1A"}
      SCAN
      Qualifier{date=558, type=SUFFIX_1}
      Qualifier{date=558, type=SUFFIX_4}
      Qualifier{date=559, type=SUFFIX_1}
      Qualifier{date=559, type=SUFFIX_4}
      

        Attachments

        1. HBASE-8930-rev6.patch
          25 kB
          Vasu Mariyala
        2. 0.96-HBASE-8930-rev3.patch
          25 kB
          Vasu Mariyala
        3. 0.94-HBASE-8930-rev2.patch
          24 kB
          Vasu Mariyala
        4. 0.96-trunk-Independent-Test-Execution.patch
          1 kB
          Vasu Mariyala
        5. 0.94-Independent-Test-Execution.patch
          0.9 kB
          Vasu Mariyala
        6. HBASE-8930-rev5.patch
          25 kB
          Vasu Mariyala
        7. 0.96-HBASE-8930-rev2.patch
          25 kB
          Vasu Mariyala
        8. HBASE-8930-rev4.patch
          26 kB
          Vasu Mariyala
        9. 0.95-HBASE-8930-rev1.patch
          26 kB
          Vasu Mariyala
        10. 0.94-HBASE-8930-rev1.patch
          24 kB
          Vasu Mariyala
        11. HBASE-8930-rev3.patch
          26 kB
          Vasu Mariyala
        12. 0.95-HBASE-8930.patch
          26 kB
          Vasu Mariyala
        13. 0.94-HBASE-8930.patch
          25 kB
          Vasu Mariyala
        14. 8930-0.94.txt
          17 kB
          Lars Hofhansl
        15. HBASE-8930-rev2.patch
          26 kB
          Vasu Mariyala
        16. HBASE-8930-rev1.patch
          26 kB
          Vasu Mariyala
        17. HBASE-8930.patch
          27 kB
          Vasu Mariyala

          Issue Links

            Activity

              People

              • Assignee:
                vmariyala Vasu Mariyala
                Reporter:
                fgaule Federico Gaule
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: