Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.0
    • Fix Version/s: 0.94.0
    • Component/s: shell
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Parent adds "raw" scan support to include delete markers and deleted rows in scan results. Would be nice it that would available in the shell to see exactly what exists in a table.

      1. 4981-v1.txt
        2 kB
        Lars Hofhansl
      2. 4981-v2.txt
        2 kB
        Lars Hofhansl
      3. 4981-v3.txt
        3 kB
        Lars Hofhansl

        Issue Links

          Activity

          Hide
          stack stack added a comment -

          +1

          What'll you all it? raw_scan?

          I want to add debug_scan and debug_get over in another issue. Not related are they?

          Show
          stack stack added a comment - +1 What'll you all it? raw_scan? I want to add debug_scan and debug_get over in another issue. Not related are they?
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Depends on what "debug_scan" means. The "raw" stuff I added in parent allow returning all KVs (including delete markers and deleted rows), is that what debug_scan does?
          (Did a quick search, but couldn't find the issue you mentioned)

          Was thinking about a syntax like RAW=>true as scan attribute. I.e.
          scan 'table',

          {VERSIONS=>10, RAW=>true}
          Show
          lhofhansl Lars Hofhansl added a comment - Depends on what "debug_scan" means. The "raw" stuff I added in parent allow returning all KVs (including delete markers and deleted rows), is that what debug_scan does? (Did a quick search, but couldn't find the issue you mentioned) Was thinking about a syntax like RAW=>true as scan attribute. I.e. scan 'table', {VERSIONS=>10, RAW=>true}
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Something like this.
          (Note that the change from get_all_columns to [] is necessary, as "raw" scans only work when no columns are specified. The behavior is identical, in both cases the scan will show all columns)

          Show
          lhofhansl Lars Hofhansl added a comment - Something like this. (Note that the change from get_all_columns to [] is necessary, as "raw" scans only work when no columns are specified. The behavior is identical, in both cases the scan will show all columns)
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Here's an example of what it would look like:

          hbase(main):001:0> scan 'x2', {RAW=>true, VERSIONS=>10}
          ROW                   COLUMN+CELL                                               
           r1                   column=f:c, timestamp=1323323611106, value=v3             
           r1                   column=f:c, timestamp=1323323609988, type=DeleteColumn    
           r1                   column=f:c, timestamp=1323323609988, value=v2             
           r1                   column=f:c, timestamp=1323323608554, value=v1             
           r2                   column=f:c, timestamp=1323323617759, value=v3             
           r2                   column=f:c, timestamp=1323323616226, value=v2             
           r2                   column=f:c, timestamp=1323323614496, value=v1             
          2 row(s) in 0.6380 seconds
          
          Show
          lhofhansl Lars Hofhansl added a comment - Here's an example of what it would look like: hbase(main):001:0> scan 'x2', {RAW=>true, VERSIONS=>10} ROW COLUMN+CELL r1 column=f:c, timestamp=1323323611106, value=v3 r1 column=f:c, timestamp=1323323609988, type=DeleteColumn r1 column=f:c, timestamp=1323323609988, value=v2 r1 column=f:c, timestamp=1323323608554, value=v1 r2 column=f:c, timestamp=1323323617759, value=v3 r2 column=f:c, timestamp=1323323616226, value=v2 r2 column=f:c, timestamp=1323323614496, value=v1 2 row(s) in 0.6380 seconds
          Hide
          lhofhansl Lars Hofhansl added a comment -

          With a family delete marker:

          hbase(main):005:0> scan 'x1', {RAW=>true, VERSIONS=>10}
          ROW                   COLUMN+CELL                                               
           r2                   column=f:, timestamp=1323323616226, type=DeleteFamily     
           r2                   column=f:c, timestamp=1323323617759, value=v3             
           r2                   column=f:c, timestamp=1323323616226, value=v2             
           r2                   column=f:c, timestamp=1323323614496, value=v1             
          2 row(s) in 0.0500 seconds
          

          Let me know what guys think, I think this is simple change and ready to go in.

          Show
          lhofhansl Lars Hofhansl added a comment - With a family delete marker: hbase(main):005:0> scan 'x1', {RAW=>true, VERSIONS=>10} ROW COLUMN+CELL r2 column=f:, timestamp=1323323616226, type=DeleteFamily r2 column=f:c, timestamp=1323323617759, value=v3 r2 column=f:c, timestamp=1323323616226, value=v2 r2 column=f:c, timestamp=1323323614496, value=v1 2 row(s) in 0.0500 seconds Let me know what guys think, I think this is simple change and ready to go in.
          Hide
          stack stack added a comment -

          This is sweet. I like passing attribute at same time. Then could have RAW => true and DEBUG => true. Issue I was talking of was HBASE-4981, a debug filter. I was thinking it would something like the following (assuming both RAW and DEBUG):

          hbase(main):005:0> scan 'x1', {RAW=>true, VERSIONS=>10}
          ROW                   COLUMN+CELL                                               
           r2                   column=f:, timestamp=1323323616226, type=DeleteFamily     
           r2                   column=f:c, timestamp=1323323617759, value=v3             
           r2                   column=f:c, timestamp=1323323616226, value=v2             
           r2                   column=f:c, timestamp=1323323614496, value=v1   
           r2                   column=f:.values_skipped_returning_this_row, timestamp=1323323614496, value=20021
           r2                   column=f:.storefiles_in_this_family, timestamp=1323323614496, value=4 
          ...          
          2 row(s) in 0.0500 seconds
          

          i.e. we'd tag on some values on end of each row result that described things like number of storefiles in the cf, and how many values we skipped returning a row result (for example, if lots of deleted values skipped, would be nice to show that somehow)

          Show
          stack stack added a comment - This is sweet. I like passing attribute at same time. Then could have RAW => true and DEBUG => true. Issue I was talking of was HBASE-4981 , a debug filter. I was thinking it would something like the following (assuming both RAW and DEBUG): hbase(main):005:0> scan 'x1', {RAW=> true , VERSIONS=>10} ROW COLUMN+CELL r2 column=f:, timestamp=1323323616226, type=DeleteFamily r2 column=f:c, timestamp=1323323617759, value=v3 r2 column=f:c, timestamp=1323323616226, value=v2 r2 column=f:c, timestamp=1323323614496, value=v1 r2 column=f:.values_skipped_returning_this_row, timestamp=1323323614496, value=20021 r2 column=f:.storefiles_in_this_family, timestamp=1323323614496, value=4 ... 2 row(s) in 0.0500 seconds i.e. we'd tag on some values on end of each row result that described things like number of storefiles in the cf, and how many values we skipped returning a row result (for example, if lots of deleted values skipped, would be nice to show that somehow)
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Nice! The debug option would be extremely helpful.

          So I'll add the RAW stuff to the scan's help blurb, and then you'll continue in HBASE-4981?

          Do I get a +1 here?

          Show
          lhofhansl Lars Hofhansl added a comment - Nice! The debug option would be extremely helpful. So I'll add the RAW stuff to the scan's help blurb, and then you'll continue in HBASE-4981 ? Do I get a +1 here?
          Hide
          stack stack added a comment -

          +1

          Show
          stack stack added a comment - +1
          Hide
          stack stack added a comment -

          Oh, do you need to mention new facility in the Scan help for shell? If you can get it in, add on commit.

          Show
          stack stack added a comment - Oh, do you need to mention new facility in the Scan help for shell? If you can get it in, add on commit.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Patch including help for scan command.

          Show
          lhofhansl Lars Hofhansl added a comment - Patch including help for scan command.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Final change that was committed.

          Show
          lhofhansl Lars Hofhansl added a comment - Final change that was committed.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Thanks for the review stack.

          Show
          lhofhansl Lars Hofhansl added a comment - Thanks for the review stack.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          This is sweet. I like passing attribute at same time. Then could have RAW => true and DEBUG => true. Issue I was talking of was HBASE-4981, a debug filter.

          Actually HBASE-4981 is this one ... You mean HBASE-4926?

          Show
          lhofhansl Lars Hofhansl added a comment - This is sweet. I like passing attribute at same time. Then could have RAW => true and DEBUG => true. Issue I was talking of was HBASE-4981 , a debug filter. Actually HBASE-4981 is this one ... You mean HBASE-4926 ?
          Hide
          stack stack added a comment -

          duh

          Show
          stack stack added a comment - duh
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK #2532 (See https://builds.apache.org/job/HBase-TRUNK/2532/)
          HBASE-4981 add raw scan support to shell (Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/ruby/hbase.rb
          • /hbase/trunk/src/main/ruby/hbase/table.rb
          • /hbase/trunk/src/main/ruby/shell/commands/scan.rb
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK #2532 (See https://builds.apache.org/job/HBase-TRUNK/2532/ ) HBASE-4981 add raw scan support to shell (Lars H) larsh : Files : /hbase/trunk/src/main/ruby/hbase.rb /hbase/trunk/src/main/ruby/hbase/table.rb /hbase/trunk/src/main/ruby/shell/commands/scan.rb
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK-security #26 (See https://builds.apache.org/job/HBase-TRUNK-security/26/)
          HBASE-4981 add raw scan support to shell (Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/ruby/hbase.rb
          • /hbase/trunk/src/main/ruby/hbase/table.rb
          • /hbase/trunk/src/main/ruby/shell/commands/scan.rb
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK-security #26 (See https://builds.apache.org/job/HBase-TRUNK-security/26/ ) HBASE-4981 add raw scan support to shell (Lars H) larsh : Files : /hbase/trunk/src/main/ruby/hbase.rb /hbase/trunk/src/main/ruby/hbase/table.rb /hbase/trunk/src/main/ruby/shell/commands/scan.rb

            People

            • Assignee:
              lhofhansl Lars Hofhansl
              Reporter:
              lhofhansl Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development