Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4947

LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards results in empty maps for the pure wildcarded column families.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 0.17.0, 0.16.1
    • Component/s: grunt
    • Labels:
      None
    • Environment:

      HDP 2.4.0
      Pig 0.15.0

    • Hadoop Flags:
      Reviewed

      Description

      LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards results in empty maps for the pure wildcarded column families.

      Here is my test scenario.
      ~~ Create test HBASE table.~~
      create 'test_sha1', '3', 'i', 'd'
      put 'test_sha1', '1', '3:name', 'youngjin'
      put 'test_sha1', '1', 'i:whatever', 'true'
      put 'test_sha1', '1', 'd:forgemeta_1_whatever', 'true'

      1. If use pure wildcarded column families then it works.
      sha1_contents_succeed = load 'hbase://test_sha1' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:*', '-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
      dump sha1_contents_succeed;
      (1,name#youngjin,whatever#true,forgemeta_1_whatever#true)

      2. If use a mix of pure wildcards and prefixed wildcards - When only one column family has prefixed wildcards - then result but only shows a prefixed wildcards column family.
      sha1_contents_no_result = load 'hbase://test_sha1' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:forgemeta_1_*', '-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
      dump sha1_contents_no_result;
      ===> (1,[],[],forgemeta_1_whatever#true)

      If use a mix of pure wildcards and prefixed wildcards - When only one column family has prefixed wildcards - then result but only shows a prefixed wildcards column family.
      sha1_contents_no_result = load 'hbase://test_sha1' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:na* i:* d:*', '-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
      (1,name#youngjin,[],[])

      If use a mix of pure wildcards and prefixed wildcards - When only one column family has prefixed wildcards - then result but only shows a prefixed wildcards column family.
      sha1_contents_no_result = load 'hbase://test_sha1' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:wh* d:*', '-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
      (1,[],whatever#true,[])

        Attachments

        1. PIG-4947-1.patch
          8 kB
          Jianyong Dai

          Activity

            People

            • Assignee:
              daijy Jianyong Dai
              Reporter:
              promiseu Youngjin
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: