HBase
  1. HBase
  2. HBASE-8642

[Snapshot] List and delete snapshot by table

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.98.0, 0.95.0, 0.95.1, 0.95.2
    • Fix Version/s: 2.0.0, 0.98.14, 1.3.0
    • Component/s: snapshots
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Support list and delete snapshots by table names.
      User scenario:
      A user wants to delete all the snapshots which were taken in January month for a table 't' where snapshot names starts with 'Jan'.

      1. 8642-trunk-0.95-v0.patch
        6 kB
        Julian Zhou
      2. 8642-trunk-0.95-v1.patch
        13 kB
        Julian Zhou
      3. 8642-trunk-0.95-v2.patch
        13 kB
        Julian Zhou
      4. HBASE-8642.patch
        22 kB
        Ashish Singhi
      5. HBASE-8642-0.98.patch
        19 kB
        Ashish Singhi
      6. HBASE-8642-v1.patch
        22 kB
        Ashish Singhi
      7. HBASE-8642-v2.patch
        22 kB
        Ashish Singhi
      8. HBASE-8642-v3.patch
        22 kB
        Ashish Singhi
      9. HBASE-8642-v4.patch
        22 kB
        Ashish Singhi

        Issue Links

          Activity

          Hide
          Matteo Bertozzi added a comment -

          overall looks good to me...
          Inside deleteSnapshotsByTable() you can use the SnapshotDescription of the for loop, instead of building a new one.
          Do you want also add the shell support?

          Show
          Matteo Bertozzi added a comment - overall looks good to me... Inside deleteSnapshotsByTable() you can use the SnapshotDescription of the for loop, instead of building a new one. Do you want also add the shell support?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585227/8642-trunk-0.95-v0.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585227/8642-trunk-0.95-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5866//console This message is automatically generated.
          Hide
          Jesse Yates added a comment -

          nit:

          Delete existing snapshots of the given table.

          to

          Delete all exisiting snapshots of a table

          Just want to call out more pointedly that it deletes all of them.

          +1, even without the above.

          Show
          Jesse Yates added a comment - nit: Delete existing snapshots of the given table. to Delete all exisiting snapshots of a table Just want to call out more pointedly that it deletes all of them. +1, even without the above.
          Hide
          Julian Zhou added a comment -

          Thanks for review, Matteo Bertozzi and Jesse Yates.
          1) for the logic inside deleteSnapshotsByTable(), I was just copying from #deleteSnapshots(final Pattern pattern), is it ok?
          2) locally applied the comment changes;
          3) for shell support, I just studied the 6353. Seems there are 2 options:
          a) add new "list_snapshots_by_table.rb" and "delete_snapshot_by_table.rb" to separate the case from regex based operation;
          b) still in "list_snapshots.rb" and "delete_snapshot.rb", but add "-table" option to branch out from regex based operation path.
          seems that "delete_snapshot.rb" has no logic for regex based operation currently, will we also support it in this stream?

          Show
          Julian Zhou added a comment - Thanks for review, Matteo Bertozzi and Jesse Yates . 1) for the logic inside deleteSnapshotsByTable(), I was just copying from #deleteSnapshots(final Pattern pattern), is it ok? 2) locally applied the comment changes; 3) for shell support, I just studied the 6353. Seems there are 2 options: a) add new "list_snapshots_by_table.rb" and "delete_snapshot_by_table.rb" to separate the case from regex based operation; b) still in "list_snapshots.rb" and "delete_snapshot.rb", but add "-table" option to branch out from regex based operation path. seems that "delete_snapshot.rb" has no logic for regex based operation currently, will we also support it in this stream?
          Hide
          Matteo Bertozzi added a comment -

          1) for the logic inside deleteSnapshotsByTable(), I was just copying from #deleteSnapshots(final Pattern pattern), is it ok?

          No is not ok, your new code should be better than the existent one
          It's not wrong, but it's just an extra copy... should be fixed also in deleteSnapshots()

          3) for shell support, I just studied the 6353. Seems there are 2 options:
          a) add new "list_snapshots_by_table.rb" and "delete_snapshot_by_table.rb" to separate the case from regex based operation;
          b) still in "list_snapshots.rb" and "delete_snapshot.rb", but add "-table" option to branch out from regex based operation path.

          Having a -table option don't seems to be inline with the current shell, I think that stuff like {{TABLE => 'name*' }} is used elsewhere (see create).. but I will pick the easy way with a new .rb

          seems that "delete_snapshot.rb" has no logic for regex based operation currently, will we also support it in this stream?

          yeah the delete snapshots is not in the shell at the moment.. I think there's a jira or a mention to open a new jira for the shell support.

          Jonathan Hsieh was against the regex delete. not sure what he thinks about this one... but at least this doesn't contain a regex for the table name.

          Show
          Matteo Bertozzi added a comment - 1) for the logic inside deleteSnapshotsByTable(), I was just copying from #deleteSnapshots(final Pattern pattern), is it ok? No is not ok, your new code should be better than the existent one It's not wrong, but it's just an extra copy... should be fixed also in deleteSnapshots() 3) for shell support, I just studied the 6353. Seems there are 2 options: a) add new "list_snapshots_by_table.rb" and "delete_snapshot_by_table.rb" to separate the case from regex based operation; b) still in "list_snapshots.rb" and "delete_snapshot.rb", but add "-table" option to branch out from regex based operation path. Having a -table option don't seems to be inline with the current shell, I think that stuff like {{TABLE => 'name*' }} is used elsewhere (see create).. but I will pick the easy way with a new .rb seems that "delete_snapshot.rb" has no logic for regex based operation currently, will we also support it in this stream? yeah the delete snapshots is not in the shell at the moment.. I think there's a jira or a mention to open a new jira for the shell support. Jonathan Hsieh was against the regex delete. not sure what he thinks about this one... but at least this doesn't contain a regex for the table name.
          Hide
          Julian Zhou added a comment -

          Hi Matteo Bertozzi, I was just adding shell support just now. Curious about "rename_snapshot" could not work? looks like HBaseAdmin has no #renameSnapshot?

          Show
          Julian Zhou added a comment - Hi Matteo Bertozzi , I was just adding shell support just now. Curious about "rename_snapshot" could not work? looks like HBaseAdmin has no #renameSnapshot?
          Hide
          Matteo Bertozzi added a comment -

          Julian Zhou yeah rename_snapshot doesn't work, take a look at the last comment on HBASE-7218.

          Show
          Matteo Bertozzi added a comment - Julian Zhou yeah rename_snapshot doesn't work, take a look at the last comment on HBASE-7218 .
          Hide
          Julian Zhou added a comment -

          Oh, thanks Matteo Bertozzi for pointing. I should do a search first.. ok, may not include into this stream?
          Another question is that why not define more functions in admin module pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex).. then shell/command/*.rb module could invoke them directly, currently, list_snapshots module has its own regex processing logic implemented? "xxx.to_java_bytes" is mandatory for ruby->java method call?

          Show
          Julian Zhou added a comment - Oh, thanks Matteo Bertozzi for pointing. I should do a search first.. ok, may not include into this stream? Another question is that why not define more functions in admin module pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex).. then shell/command/*.rb module could invoke them directly, currently, list_snapshots module has its own regex processing logic implemented? "xxx.to_java_bytes" is mandatory for ruby->java method call?
          Hide
          Matteo Bertozzi added a comment -

          Another question is that why not define more functions in admin module pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex).. then shell/command/*.rb module could invoke them directly, currently, list_snapshots module has its own regex processing logic implemented? "xxx.to_java_bytes" is mandatory for ruby->java method call?

          No particular reason, the admin.listSnapshots(regex) & co were added later, and the shell was not updated to use them. and since the regex stuff is just a loop with an if there's no real advantage of having it in the admin, aside saving you from write 3 extra lines of code.
          Jon was suggesting to have extra logic in the shell for stuff like delete with regex, where you may want to add an "are you sure?" check in front.

          Show
          Matteo Bertozzi added a comment - Another question is that why not define more functions in admin module pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex).. then shell/command/*.rb module could invoke them directly, currently, list_snapshots module has its own regex processing logic implemented? "xxx.to_java_bytes" is mandatory for ruby->java method call? No particular reason, the admin.listSnapshots(regex) & co were added later, and the shell was not updated to use them. and since the regex stuff is just a loop with an if there's no real advantage of having it in the admin, aside saving you from write 3 extra lines of code. Jon was suggesting to have extra logic in the shell for stuff like delete with regex, where you may want to add an "are you sure?" check in front.
          Hide
          Julian Zhou added a comment -

          Hi, Matteo Bertozzi, applied the changes. Could you help review? Thanks~

          Show
          Julian Zhou added a comment - Hi, Matteo Bertozzi , applied the changes. Could you help review? Thanks~
          Hide
          Matteo Bertozzi added a comment -

          v1 looks good to me

          Show
          Matteo Bertozzi added a comment - v1 looks good to me
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585434/8642-trunk-0.95-v1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585434/8642-trunk-0.95-v1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5886//console This message is automatically generated.
          Hide
          Matteo Bertozzi added a comment -

          I'll commit this one tomorrow, if there are no objections.

          Show
          Matteo Bertozzi added a comment - I'll commit this one tomorrow, if there are no objections.
          Hide
          Jonathan Hsieh added a comment -

          The idea of a delete by table name commands seems reasonable. This might get confusing if we have a rename table functionality but at the moment we don't.

          Show
          Jonathan Hsieh added a comment - The idea of a delete by table name commands seems reasonable. This might get confusing if we have a rename table functionality but at the moment we don't.
          Hide
          Ted Yu added a comment -
          +    def delete_snapshots_by_table(table)
          

          Can the new commands be made shorter ?

          Show
          Ted Yu added a comment - + def delete_snapshots_by_table(table) Can the new commands be made shorter ?
          Hide
          Matteo Bertozzi added a comment -

          this might get confusing if we have a rename table functionality but at the moment we don't.

          right... we sort of have the rename table mentioned in the docbook...
          since we don't have a proper naming system, my guess is that we should just alert the user... if you rename the table snapshots are not updated with the new table name...

          Show
          Matteo Bertozzi added a comment - this might get confusing if we have a rename table functionality but at the moment we don't. right... we sort of have the rename table mentioned in the docbook... since we don't have a proper naming system, my guess is that we should just alert the user... if you rename the table snapshots are not updated with the new table name...
          Hide
          Julian Zhou added a comment -

          Table rename is via cloning snapshot currently (so we rename the table this way, pairwisely we update all mapped snapshots?)? How about the claiming is that listing/deleting snapshot by table name is based on current table name available?

          For shorter cmd, how about "delete_table_snapshot"/"list_table_snapshot"?

          Show
          Julian Zhou added a comment - Table rename is via cloning snapshot currently (so we rename the table this way, pairwisely we update all mapped snapshots?)? How about the claiming is that listing/deleting snapshot by table name is based on current table name available? For shorter cmd, how about "delete_table_snapshot"/"list_table_snapshot"?
          Hide
          Julian Zhou added a comment -

          Yeah, RDBMS has object (database, tablespace, table, column, etc.) id in catalog, which make external name changable and transparent to system. Is it possible to using id (shorter) instead of original name bytes in ROOT/META? Just a question, this involve lots of changes to current HBase.

          Show
          Julian Zhou added a comment - Yeah, RDBMS has object (database, tablespace, table, column, etc.) id in catalog, which make external name changable and transparent to system. Is it possible to using id (shorter) instead of original name bytes in ROOT/META? Just a question, this involve lots of changes to current HBase.
          Hide
          Matteo Bertozzi added a comment -

          Table rename is via cloning snapshot currently (so we rename the table this way, pairwisely we update all mapped snapshots?)?

          No we don't update the other snapshots info... so they have the old table name...

          How about the claiming is that listing/deleting snapshot by table name is based on current table name available?

          I don't really understand this one... at the moment if you list by table name, you get also the snapshots of the renamed table.

          Yeah, RDBMS has object (database, tablespace, table, column, etc.) id in catalog, which make external name changable and transparent to system. Is it possible to using id (shorter) instead of original name bytes in ROOT/META? Just a question, this involve lots of changes to current HBase.

          Take a look at the last slide of HBASE-7806 pdf

          Show
          Matteo Bertozzi added a comment - Table rename is via cloning snapshot currently (so we rename the table this way, pairwisely we update all mapped snapshots?)? No we don't update the other snapshots info... so they have the old table name... How about the claiming is that listing/deleting snapshot by table name is based on current table name available? I don't really understand this one... at the moment if you list by table name, you get also the snapshots of the renamed table. Yeah, RDBMS has object (database, tablespace, table, column, etc.) id in catalog, which make external name changable and transparent to system. Is it possible to using id (shorter) instead of original name bytes in ROOT/META? Just a question, this involve lots of changes to current HBase. Take a look at the last slide of HBASE-7806 pdf
          Hide
          Ted Yu added a comment -

          how about "delete_table_snapshot"/"list_table_snapshot"?

          I think "delete_table_snapshots"/"list_table_snapshots" are good.

          Show
          Ted Yu added a comment - how about "delete_table_snapshot"/"list_table_snapshot"? I think "delete_table_snapshots"/"list_table_snapshots" are good.
          Hide
          Julian Zhou added a comment -

          Hi, Matteo Bertozzi, I see that last slide of HBASE-7806 for future, is that to seperate .data and .meta to enable possible file sharing among tables. One byte (256 items could be attached) for each node anchor under .data tree. This could be a new catalog system for hbase.

          Show
          Julian Zhou added a comment - Hi, Matteo Bertozzi , I see that last slide of HBASE-7806 for future, is that to seperate .data and .meta to enable possible file sharing among tables. One byte (256 items could be attached) for each node anchor under .data tree. This could be a new catalog system for hbase.
          Hide
          Matteo Bertozzi added a comment -

          is that to seperate data and meta to enable possible file sharing among tables. This could be a new catalog system for hbase.

          Yeah that's the idea... it not just file sharing between table/snapshots is also the ability to switch to managing blocks directly and avoid to rewrite everything during compactions.

          One byte (256 items could be attached) for each node anchor under .data tree.

          Not sure what you mean with this... The file placement using the first bytes of the hash/name is just to avoid the problem of too many files in a directory that old filesystems have, and a different way to store stuff removing the knowledge of table/region/cf that today is hardcoded. This doesn't really have effect on all the capabilities that you'll gain from the richer metadata system.

          (This stuff is starting being off topic here, since is not related 100% with this jira, if you've other question/suggestions/... just write it on HBASE-7806)

          Show
          Matteo Bertozzi added a comment - is that to seperate data and meta to enable possible file sharing among tables. This could be a new catalog system for hbase. Yeah that's the idea... it not just file sharing between table/snapshots is also the ability to switch to managing blocks directly and avoid to rewrite everything during compactions. One byte (256 items could be attached) for each node anchor under .data tree. Not sure what you mean with this... The file placement using the first bytes of the hash/name is just to avoid the problem of too many files in a directory that old filesystems have, and a different way to store stuff removing the knowledge of table/region/cf that today is hardcoded. This doesn't really have effect on all the capabilities that you'll gain from the richer metadata system. (This stuff is starting being off topic here, since is not related 100% with this jira, if you've other question/suggestions/... just write it on HBASE-7806 )
          Hide
          Julian Zhou added a comment -

          Updated with new ruby module name. Could we go ahead with this?

          Show
          Julian Zhou added a comment - Updated with new ruby module name. Could we go ahead with this?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5907//console This message is automatically generated.
          Hide
          stack added a comment -

          Matteo Bertozzi You good w/ the last patch boss? If so, commit? I took a look and looks good to me. Thanks boss.

          Show
          stack added a comment - Matteo Bertozzi You good w/ the last patch boss? If so, commit? I took a look and looks good to me. Thanks boss.
          Hide
          stack added a comment -

          Matteo Bertozzi You see above boss?

          Show
          stack added a comment - Matteo Bertozzi You see above boss?
          Hide
          Matteo Bertozzi added a comment -

          patch looks good to me, now doesn't apply cleanly but the fix is trivial.
          my main concern with this feature is related to the rename semantic, where you rename a table and listing/deleting with the old rename results in including the snapshots made on before the rename table. Which may be fine, since may be considered not totally wrong and we don't have rename support in the core.

          so, I'm +1 on the code and +0 on the merge, I'll let you decide.

          Show
          Matteo Bertozzi added a comment - patch looks good to me, now doesn't apply cleanly but the fix is trivial. my main concern with this feature is related to the rename semantic, where you rename a table and listing/deleting with the old rename results in including the snapshots made on before the rename table. Which may be fine, since may be considered not totally wrong and we don't have rename support in the core. so, I'm +1 on the code and +0 on the merge, I'll let you decide.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8184//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8184//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch
          against trunk revision .
          ATTACHMENT ID: 12585796

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10772//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch against trunk revision . ATTACHMENT ID: 12585796 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10772//console This message is automatically generated.
          Hide
          Enis Soztutar added a comment -

          Lets do this. My only concern is that all new API should only be using TableName args, and not byte[] or String table name args.

          Show
          Enis Soztutar added a comment - Lets do this. My only concern is that all new API should only be using TableName args, and not byte[] or String table name args.
          Hide
          Sean Busbey added a comment -

          bump. Julian Zhou are you still interested in working on this?

          Show
          Sean Busbey added a comment - bump. Julian Zhou are you still interested in working on this?
          Hide
          Ashish Singhi added a comment -

          From Julian Zhou activity stream looks like he is not active here from some time.
          I will try to take this over the finish line.

          Show
          Ashish Singhi added a comment - From Julian Zhou activity stream looks like he is not active here from some time. I will try to take this over the finish line.
          Hide
          Ashish Singhi added a comment -

          updated the description with user scenario.

          Show
          Ashish Singhi added a comment - updated the description with user scenario.
          Hide
          Ashish Singhi added a comment -

          Added two commands list_table_snapshots and delete_table_snapshots which takes table name and snapshot name. For shell command snapshot name is optional, if it is not passed then all the snapshots matching the table name regex will be listed (and eligible for delete if executing delete_table_snapshots command).

          A user wants to delete all the snapshots which were taken in January month for a table 't' where snapshot names starts with 'Jan'.

          This can be done now by executing below command

          hbase(main):028:0> delete_table_snapshots 't', 'Jan.*'

          Verified shell commands manually everything seems ok.

          Will move this jira to Patch Available state once Hadoop QA for master branch becomes ok.

          Please review.

          Show
          Ashish Singhi added a comment - Added two commands list_table_snapshots and delete_table_snapshots which takes table name and snapshot name. For shell command snapshot name is optional, if it is not passed then all the snapshots matching the table name regex will be listed (and eligible for delete if executing delete_table_snapshots command). A user wants to delete all the snapshots which were taken in January month for a table 't' where snapshot names starts with 'Jan'. This can be done now by executing below command hbase(main):028:0> delete_table_snapshots 't', 'Jan.*' Verified shell commands manually everything seems ok. Will move this jira to Patch Available state once Hadoop QA for master branch becomes ok. Please review.
          Hide
          Ashish Singhi added a comment -

          Attached RB link

          Show
          Ashish Singhi added a comment - Attached RB link
          Hide
          Sean Busbey added a comment -

          QA Bot for master should be fine now, moving to Patch Available.

          Show
          Sean Busbey added a comment - QA Bot for master should be fine now, moving to Patch Available.
          Hide
          Ashish Singhi added a comment -

          Thanks.

          Show
          Ashish Singhi added a comment - Thanks.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12741604/HBASE-8642.patch
          against master branch at commit 578adca6ee961f558cd2b2246156f9822cf4f7a2.
          ATTACHMENT ID: 12741604

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "#

          {successfullyDeleted}

          snapshots successfully deleted." unless successfullyDeleted == 0

          -1 site. The patch appears to cause mvn post-site goal to fail.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.util.TestProcessBasedCluster
          org.apache.hadoop.hbase.mapreduce.TestImportExport
          org.apache.hadoop.hbase.TestRegionRebalancing

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12741604/HBASE-8642.patch against master branch at commit 578adca6ee961f558cd2b2246156f9822cf4f7a2. ATTACHMENT ID: 12741604 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "# {successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 -1 site . The patch appears to cause mvn post-site goal to fail. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.util.TestProcessBasedCluster org.apache.hadoop.hbase.mapreduce.TestImportExport org.apache.hadoop.hbase.TestRegionRebalancing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14547//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          Unknown macro: {tableNameregex.to_s}

          and the snapshot name regular expression #

          Unknown macro: {snapshotNameRegex.to_s}

          " if count == 0
          + puts "#

          Unknown macro: {successfullyDeleted}

          snapshots successfully deleted." unless successfullyDeleted == 0

          This is related to ruby script. I think we don't fix lineLengths warning for ruby files. Correct if I am wrong ? I tried formatting but the output on the console doesn't looks good.

          Show
          Ashish Singhi added a comment - -1 lineLengths. The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # Unknown macro: {tableNameregex.to_s} and the snapshot name regular expression # Unknown macro: {snapshotNameRegex.to_s} " if count == 0 + puts "# Unknown macro: {successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 This is related to ruby script. I think we don't fix lineLengths warning for ruby files. Correct if I am wrong ? I tried formatting but the output on the console doesn't looks good.
          Hide
          Ashish Singhi added a comment -

          Thanks Matteo Bertozzi for the review.
          Addressed your comment from RB.

          I have not fixed the lineLengths warning for the ruby script as I am not sure how to do it and the test failures were not related to the patch.

          Please review.

          Show
          Ashish Singhi added a comment - Thanks Matteo Bertozzi for the review. Addressed your comment from RB. I have not fixed the lineLengths warning for the ruby script as I am not sure how to do it and the test failures were not related to the patch. Please review.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12741823/HBASE-8642-v1.patch
          against master branch at commit 2df3236a4eee48bf723213a7c4ff3d29c832c8cf.
          ATTACHMENT ID: 12741823

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "\nDelete the above #

          {count}

          snapshots (y/n)? \nNOTE: Snapshot(s) matching the given regular expressions and taken after the above list is displayed will be also deleted." unless count == 0
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "#

          {successfullyDeleted}

          snapshots successfully deleted." unless successfullyDeleted == 0

          +1 site. The mvn post-site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.TestRegionRebalancing

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12741823/HBASE-8642-v1.patch against master branch at commit 2df3236a4eee48bf723213a7c4ff3d29c832c8cf. ATTACHMENT ID: 12741823 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "\nDelete the above # {count} snapshots (y/n)? \nNOTE: Snapshot(s) matching the given regular expressions and taken after the above list is displayed will be also deleted." unless count == 0 + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "# {successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 +1 site . The mvn post-site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestRegionRebalancing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14566//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          Patch addressing Matteo's concern.
          Please review.

          Show
          Ashish Singhi added a comment - Patch addressing Matteo's concern. Please review.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12742324/HBASE-8642-v2.patch
          against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06.
          ATTACHMENT ID: 12742324

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "#

          {successfullyDeleted}

          snapshots successfully deleted." unless successfullyDeleted == 0

          +1 site. The mvn post-site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12742324/HBASE-8642-v2.patch against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06. ATTACHMENT ID: 12742324 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "# {successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 +1 site . The mvn post-site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14593//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          Matteo Bertozzi can you please review the latest patch.

          Show
          Ashish Singhi added a comment - Matteo Bertozzi can you please review the latest patch.
          Hide
          Anoop Sam John added a comment -

          Latest patch looks good.. May be one of the examples in the shell commands can use a ns:tableName ?
          Hope u have tested shell commands with all possible combinations.

          Show
          Anoop Sam John added a comment - Latest patch looks good.. May be one of the examples in the shell commands can use a ns:tableName ? Hope u have tested shell commands with all possible combinations.
          Hide
          Ashish Singhi added a comment -

          Thanks Anoop for the review. Added one example with namespace in v3 patch.

          Hope u have tested shell commands with all possible combinations.

          Following are the combinations I have tested with their output.

          hbase(main):015:0> list
          TABLE
          ns:t
          t
          t1
          u
          4 row(s) in 0.0330 seconds
          
          => ["ns:t", "t", "t1", "u"]
          
          hbase(main):016:0> list_snapshots
          
          SNAPSHOT                                             TABLE + CREATION TIME
           as2                                                 t1 (Wed Jul 01 18:16:07 +0530 2015)
           as3                                                 ns:t (Wed Jul 01 18:16:26 +0530 2015)
           p                                                   t (Wed Jul 01 18:15:54 +0530 2015)
           q                                                   u (Wed Jul 01 18:16:42 +0530 2015)
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
           s4                                                  u (Wed Jul 01 18:16:47 +0530 2015)
          9 row(s) in 0.0780 seconds
          
          => ["as2", "as3", "p", "q", "s", "s1", "s2", "s3", "s4"]
          
          hbase(main):017:0> delete_table_snapshots '.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as2                                                 t1 (Wed Jul 01 18:16:07 +0530 2015)
           as3                                                 ns:t (Wed Jul 01 18:16:26 +0530 2015)
           p                                                   t (Wed Jul 01 18:15:54 +0530 2015)
           q                                                   u (Wed Jul 01 18:16:42 +0530 2015)
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
           s4                                                  u (Wed Jul 01 18:16:47 +0530 2015)
          
          Delete the above 9 snapshots (y/n)?
          n
          
          hbase(main):018:0> delete_table_snapshots '.*', '.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as2                                                 t1 (Wed Jul 01 18:16:07 +0530 2015)
           as3                                                 ns:t (Wed Jul 01 18:16:26 +0530 2015)
           p                                                   t (Wed Jul 01 18:15:54 +0530 2015)
           q                                                   u (Wed Jul 01 18:16:42 +0530 2015)
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
           s4                                                  u (Wed Jul 01 18:16:47 +0530 2015)
          
          Delete the above 9 snapshots (y/n)?
          n
          
          hbase(main):022:0> delete_table_snapshots 't.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as2                                                 t1 (Wed Jul 01 18:16:07 +0530 2015)
           p                                                   t (Wed Jul 01 18:15:54 +0530 2015)
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
          
          Delete the above 5 snapshots (y/n)?
          n
          
          hbase(main):023:0> delete_table_snapshots 't.*', 's.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
          
          Delete the above 3 snapshots (y/n)?
          n
          
          hbase(main):024:0> delete_table_snapshots 'ns:t.*', 's.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
          
          Delete the above 1 snapshots (y/n)?
          n
          
          hbase(main):025:0> delete_table_snapshots 'ns:t.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as3                                                 ns:t (Wed Jul 01 18:16:26 +0530 2015)
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
          
          Delete the above 2 snapshots (y/n)?
          n
          
          hbase(main):026:0> delete_table_snapshots 'ns:t.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as3                                                 ns:t (Wed Jul 01 18:16:26 +0530 2015)
           s3                                                  ns:t (Wed Jul 01 18:16:22 +0530 2015)
          
          Delete the above 2 snapshots (y/n)?
          y
          0 row(s) in 0.1060 seconds
          2 snapshots successfully deleted.
          
          hbase(main):027:0> delete_table_snapshots '.*', 's.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           s                                                   t (Wed Jul 01 18:15:45 +0530 2015)
           s1                                                  t (Wed Jul 01 18:15:48 +0530 2015)
           s2                                                  t1 (Wed Jul 01 18:16:01 +0530 2015)
           s4                                                  u (Wed Jul 01 18:16:47 +0530 2015)
          
          Delete the above 4 snapshots (y/n)?
          y
          0 row(s) in 0.0910 seconds
          4 snapshots successfully deleted.
          
          hbase(main):028:0> delete_table_snapshots '.*', '.*'
          SNAPSHOT                                             TABLE + CREATION TIME
           as2                                                 t1 (Wed Jul 01 18:16:07 +0530 2015)
           p                                                   t (Wed Jul 01 18:15:54 +0530 2015)
           q                                                   u (Wed Jul 01 18:16:42 +0530 2015)
          
          Delete the above 3 snapshots (y/n)?
          y
          0 row(s) in 0.0720 seconds
          3 snapshots successfully deleted.
          
          hbase(main):030:0> delete_table_snapshots '.*'
          SNAPSHOT                                             TABLE + CREATION TIME
          No snapshots matched the table name regular expression .* and the snapshot name regular expression .*
          
          hbase(main):032:0> delete_table_snapshots 't'
          SNAPSHOT                                             TABLE + CREATION TIME
          No snapshots matched the table name regular expression t and the snapshot name regular expression .*
          
          hbase(main):033:0> delete_table_snapshots
          
          ERROR: wrong number of arguments (0 for 1)
          
          Here is some help for this command:
          Delete all of the snapshots matching the given table name regular expression
          and snapshot name regular expression.
          By default snapshot name regular expression will delete all the snapshots of the
          matching table name regular expression.
          
          Examples:
            hbase> delete_table_snapshots 'tableName'
            hbase> delete_table_snapshots 'tableName.*'
            hbase> delete_table_snapshots 'tableName' 'snapshotName'
            hbase> delete_table_snapshots 'tableName' 'snapshotName.*'
            hbase> delete_table_snapshots 'tableName.*' 'snapshotName.*'
            hbase> delete_table_snapshots 'ns:tableName.*' 'snapshotName.*'
          
          Show
          Ashish Singhi added a comment - Thanks Anoop for the review. Added one example with namespace in v3 patch. Hope u have tested shell commands with all possible combinations. Following are the combinations I have tested with their output. hbase(main):015:0> list TABLE ns:t t t1 u 4 row(s) in 0.0330 seconds => ["ns:t", "t", "t1", "u"] hbase(main):016:0> list_snapshots SNAPSHOT TABLE + CREATION TIME as2 t1 (Wed Jul 01 18:16:07 +0530 2015) as3 ns:t (Wed Jul 01 18:16:26 +0530 2015) p t (Wed Jul 01 18:15:54 +0530 2015) q u (Wed Jul 01 18:16:42 +0530 2015) s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) s4 u (Wed Jul 01 18:16:47 +0530 2015) 9 row(s) in 0.0780 seconds => ["as2", "as3", "p", "q", "s", "s1", "s2", "s3", "s4"] hbase(main):017:0> delete_table_snapshots '.*' SNAPSHOT TABLE + CREATION TIME as2 t1 (Wed Jul 01 18:16:07 +0530 2015) as3 ns:t (Wed Jul 01 18:16:26 +0530 2015) p t (Wed Jul 01 18:15:54 +0530 2015) q u (Wed Jul 01 18:16:42 +0530 2015) s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) s4 u (Wed Jul 01 18:16:47 +0530 2015) Delete the above 9 snapshots (y/n)? n hbase(main):018:0> delete_table_snapshots '.*', '.*' SNAPSHOT TABLE + CREATION TIME as2 t1 (Wed Jul 01 18:16:07 +0530 2015) as3 ns:t (Wed Jul 01 18:16:26 +0530 2015) p t (Wed Jul 01 18:15:54 +0530 2015) q u (Wed Jul 01 18:16:42 +0530 2015) s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) s4 u (Wed Jul 01 18:16:47 +0530 2015) Delete the above 9 snapshots (y/n)? n hbase(main):022:0> delete_table_snapshots 't.*' SNAPSHOT TABLE + CREATION TIME as2 t1 (Wed Jul 01 18:16:07 +0530 2015) p t (Wed Jul 01 18:15:54 +0530 2015) s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) Delete the above 5 snapshots (y/n)? n hbase(main):023:0> delete_table_snapshots 't.*', 's.*' SNAPSHOT TABLE + CREATION TIME s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) Delete the above 3 snapshots (y/n)? n hbase(main):024:0> delete_table_snapshots 'ns:t.*', 's.*' SNAPSHOT TABLE + CREATION TIME s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) Delete the above 1 snapshots (y/n)? n hbase(main):025:0> delete_table_snapshots 'ns:t.*' SNAPSHOT TABLE + CREATION TIME as3 ns:t (Wed Jul 01 18:16:26 +0530 2015) s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) Delete the above 2 snapshots (y/n)? n hbase(main):026:0> delete_table_snapshots 'ns:t.*' SNAPSHOT TABLE + CREATION TIME as3 ns:t (Wed Jul 01 18:16:26 +0530 2015) s3 ns:t (Wed Jul 01 18:16:22 +0530 2015) Delete the above 2 snapshots (y/n)? y 0 row(s) in 0.1060 seconds 2 snapshots successfully deleted. hbase(main):027:0> delete_table_snapshots '.*', 's.*' SNAPSHOT TABLE + CREATION TIME s t (Wed Jul 01 18:15:45 +0530 2015) s1 t (Wed Jul 01 18:15:48 +0530 2015) s2 t1 (Wed Jul 01 18:16:01 +0530 2015) s4 u (Wed Jul 01 18:16:47 +0530 2015) Delete the above 4 snapshots (y/n)? y 0 row(s) in 0.0910 seconds 4 snapshots successfully deleted. hbase(main):028:0> delete_table_snapshots '.*', '.*' SNAPSHOT TABLE + CREATION TIME as2 t1 (Wed Jul 01 18:16:07 +0530 2015) p t (Wed Jul 01 18:15:54 +0530 2015) q u (Wed Jul 01 18:16:42 +0530 2015) Delete the above 3 snapshots (y/n)? y 0 row(s) in 0.0720 seconds 3 snapshots successfully deleted. hbase(main):030:0> delete_table_snapshots '.*' SNAPSHOT TABLE + CREATION TIME No snapshots matched the table name regular expression .* and the snapshot name regular expression .* hbase(main):032:0> delete_table_snapshots 't' SNAPSHOT TABLE + CREATION TIME No snapshots matched the table name regular expression t and the snapshot name regular expression .* hbase(main):033:0> delete_table_snapshots ERROR: wrong number of arguments (0 for 1) Here is some help for this command: Delete all of the snapshots matching the given table name regular expression and snapshot name regular expression. By default snapshot name regular expression will delete all the snapshots of the matching table name regular expression. Examples: hbase> delete_table_snapshots 'tableName' hbase> delete_table_snapshots 'tableName.*' hbase> delete_table_snapshots 'tableName' 'snapshotName' hbase> delete_table_snapshots 'tableName' 'snapshotName.*' hbase> delete_table_snapshots 'tableName.*' 'snapshotName.*' hbase> delete_table_snapshots 'ns:tableName.*' 'snapshotName.*'
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12743016/HBASE-8642-v3.patch
          against master branch at commit 85c278a6a8b25ff86e22c254ffec35e945cd7c66.
          ATTACHMENT ID: 12743016

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "#

          {successfullyDeleted}

          snapshots successfully deleted." unless successfullyDeleted == 0

          +1 site. The mvn post-site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12743016/HBASE-8642-v3.patch against master branch at commit 85c278a6a8b25ff86e22c254ffec35e945cd7c66. ATTACHMENT ID: 12743016 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "# {successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 +1 site . The mvn post-site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14637//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          Patch addressing Matteo comment from RB.
          Also ensured that there is some way for a user to know which set of snapshots got successfully deleted or failed.

          Please review.

          Show
          Ashish Singhi added a comment - Patch addressing Matteo comment from RB. Also ensured that there is some way for a user to know which set of snapshots got successfully deleted or failed. Please review.
          Hide
          Ashish Singhi added a comment -

          some way for a user to know which set of snapshots got successfully deleted or failed.

          This is how it will be displayed on the console.

          hbase(main):002:0> delete_table_snapshots 't', '.*'
          SNAPSHOT                              TABLE + CREATION TIME                                                                                     
           a                                    t (Fri Jul 03 19:40:17 +0530 2015)                                                                        
           s                                    t (Fri Jul 03 19:40:19 +0530 2015)                                                                        
           s1                                   t (Fri Jul 03 19:40:21 +0530 2015)                                                                        
          
          Delete the above 3 snapshots (y/n)?
          y
          Successfully deleted snapshot: a
          
          Failed to delete snapshot: s, due to below exception,
          org.apache.hadoop.hbase.snapshot.SnapshotDoesNotExistException: org.apache.hadoop.hbase.snapshot.SnapshotDoesNotExistException: Snapshot 's' doesn't exist on the filesystem
          	at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.deleteSnapshot(SnapshotManager.java:282)
          	at org.apache.hadoop.hbase.master.MasterRpcServices.deleteSnapshot(MasterRpcServices.java:464)
          	at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:49860)
          	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2132)
          	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
          	at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
          	at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
          	at java.lang.Thread.run(Thread.java:745)
          
          Successfully deleted snapshot: s1
          
          0 row(s) in 0.0460 seconds
          
          
          Show
          Ashish Singhi added a comment - some way for a user to know which set of snapshots got successfully deleted or failed. This is how it will be displayed on the console. hbase(main):002:0> delete_table_snapshots 't', '.*' SNAPSHOT TABLE + CREATION TIME a t (Fri Jul 03 19:40:17 +0530 2015) s t (Fri Jul 03 19:40:19 +0530 2015) s1 t (Fri Jul 03 19:40:21 +0530 2015) Delete the above 3 snapshots (y/n)? y Successfully deleted snapshot: a Failed to delete snapshot: s, due to below exception, org.apache.hadoop.hbase.snapshot.SnapshotDoesNotExistException: org.apache.hadoop.hbase.snapshot.SnapshotDoesNotExistException: Snapshot 's' doesn't exist on the filesystem at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.deleteSnapshot(SnapshotManager.java:282) at org.apache.hadoop.hbase.master.MasterRpcServices.deleteSnapshot(MasterRpcServices.java:464) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:49860) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2132) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:745) Successfully deleted snapshot: s1 0 row(s) in 0.0460 seconds
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12743520/HBASE-8642-v4.patch
          against master branch at commit e640f1e76af8f32015f475629610da127897f01e.
          ATTACHMENT ID: 12743520

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "Failed to delete snapshot: #

          {deleteSnapshot.getName}

          , due to below exception,\n" + $!

          +1 site. The mvn post-site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//testReport/
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//artifact/patchprocess/checkstyle-aggregate.html

          Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//artifact/patchprocess/patchJavadocWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12743520/HBASE-8642-v4.patch against master branch at commit e640f1e76af8f32015f475629610da127897f01e. ATTACHMENT ID: 12743520 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors -1 findbugs . The patch appears to cause Findbugs (version 2.0.3) to fail. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "Failed to delete snapshot: # {deleteSnapshot.getName} , due to below exception,\n" + $! +1 site . The mvn post-site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//testReport/ Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14657//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages
          -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail.

          Not related to the patch. Javadoc warning I have seen reported in other Hadoop QA report as well.

          -1 core tests. The patch failed these unit tests:

          $:$ python findHangingTests.py https://builds.apache.org/job/PreCommit-HBASE-Build/14657//consoleFull
          Fetching the console output from the URL
          Printing hanging tests
          Hanging test : org.apache.hadoop.hbase.util.TestDrainBarrier
          Printing Failing tests
          

          Not related to the patch.

          Show
          Ashish Singhi added a comment - -1 javadoc. The javadoc tool appears to have generated 1 warning messages -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail. Not related to the patch. Javadoc warning I have seen reported in other Hadoop QA report as well. -1 core tests. The patch failed these unit tests: $:$ python findHangingTests.py https://builds.apache.org/job/PreCommit-HBASE-Build/14657//consoleFull Fetching the console output from the URL Printing hanging tests Hanging test : org.apache.hadoop.hbase.util.TestDrainBarrier Printing Failing tests Not related to the patch.
          Hide
          Ashish Singhi added a comment -

          Patch for 0.98 branch.

          Show
          Ashish Singhi added a comment - Patch for 0.98 branch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12743709/HBASE-8642-0.98.patch
          against 0.98 branch at commit 7d3456d8fd027e252b1da7578e943f146626135d.
          ATTACHMENT ID: 12743709

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 24 warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + puts "No snapshots matched the table name regular expression #

          {tableNameregex.to_s}

          and the snapshot name regular expression #

          {snapshotNameRegex.to_s}

          " if count == 0
          + puts "Failed to delete snapshot: #

          {deleteSnapshot.getName}

          , due to below exception,\n" + $!

          +1 site. The mvn post-site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:

          -1 core zombie tests. There are 2 zombie test(s): at org.apache.sling.testing.tools.sling.BundlesInstaller.startAllBundles(BundlesInstaller.java:132)
          at org.apache.sling.testing.tools.sling.SlingTestBase.installAdditionalBundles(SlingTestBase.java:186)
          at org.apache.sling.testing.tools.sling.SlingTestBase.startServerIfNeeded(SlingTestBase.java:153)
          at org.apache.sling.testing.tools.sling.SlingTestBase.getRequestBuilder(SlingTestBase.java:207)
          at org.apache.sling.resourceaccesssecurity.it.ResourceAccessSecurityTestBase.testRead(ResourceAccessSecurityTestBase.java:63)
          at org.apache.sling.resourceaccesssecurity.it.UnsecuredProviderResourceAccessSecurityTest.testCantReadOnlyApplicationAccessGatePresent(UnsecuredProviderResourceAccessSecurityTest.java:59)

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/checkstyle-aggregate.html

          Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/patchJavadocWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12743709/HBASE-8642-0.98.patch against 0.98 branch at commit 7d3456d8fd027e252b1da7578e943f146626135d. ATTACHMENT ID: 12743709 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. -1 javadoc . The javadoc tool appears to have generated 24 warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + puts "No snapshots matched the table name regular expression # {tableNameregex.to_s} and the snapshot name regular expression # {snapshotNameRegex.to_s} " if count == 0 + puts "Failed to delete snapshot: # {deleteSnapshot.getName} , due to below exception,\n" + $! +1 site . The mvn post-site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 2 zombie test(s): at org.apache.sling.testing.tools.sling.BundlesInstaller.startAllBundles(BundlesInstaller.java:132) at org.apache.sling.testing.tools.sling.SlingTestBase.installAdditionalBundles(SlingTestBase.java:186) at org.apache.sling.testing.tools.sling.SlingTestBase.startServerIfNeeded(SlingTestBase.java:153) at org.apache.sling.testing.tools.sling.SlingTestBase.getRequestBuilder(SlingTestBase.java:207) at org.apache.sling.resourceaccesssecurity.it.ResourceAccessSecurityTestBase.testRead(ResourceAccessSecurityTestBase.java:63) at org.apache.sling.resourceaccesssecurity.it.UnsecuredProviderResourceAccessSecurityTest.testCantReadOnlyApplicationAccessGatePresent(UnsecuredProviderResourceAccessSecurityTest.java:59) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14675//console This message is automatically generated.
          Hide
          Ashish Singhi added a comment -

          Ping for more reviews!

          Show
          Ashish Singhi added a comment - Ping for more reviews!
          Hide
          Ashish Singhi added a comment -

          Matteo Bertozzi can you please review the latest version of the patch.

          Show
          Ashish Singhi added a comment - Matteo Bertozzi can you please review the latest version of the patch.
          Hide
          Andrew Purtell added a comment -

          It looks like all review feedback has been addressed. Let me look at this later today for potential inclusion in 0.98. We can take enhancements there as long as there are no incompatible changes and no compilation issues with supported Java and Hadoop versions for that release line in the support matrix. If so I will commit back through branch-1 but excluding current 1.x release lines as indicated by fix versions.

          Show
          Andrew Purtell added a comment - It looks like all review feedback has been addressed. Let me look at this later today for potential inclusion in 0.98. We can take enhancements there as long as there are no incompatible changes and no compilation issues with supported Java and Hadoop versions for that release line in the support matrix. If so I will commit back through branch-1 but excluding current 1.x release lines as indicated by fix versions.
          Hide
          Andrew Purtell added a comment -

          I don't see any compat issues. We're missing shell tests for the new list_table_snapshots and delete_table_snapshots shell commands but do have new tests for the underlying functional additions to HBaseAdmin. No issues found wrt the older stuff we need to support with 0.98. Running tests locally before commit.

          Show
          Andrew Purtell added a comment - I don't see any compat issues. We're missing shell tests for the new list_table_snapshots and delete_table_snapshots shell commands but do have new tests for the underlying functional additions to HBaseAdmin. No issues found wrt the older stuff we need to support with 0.98. Running tests locally before commit.
          Hide
          Andrew Purtell added a comment -

          Pushed to 0.98, branch-1, and master.

          I also tested the new commands by creating three snapshots, listing them, then deleting them in one go with delete_table_snapshots.

          It's a bit odd that the delete_table_snapshots command asks for confirmation where others do not. That is reasonable given how destructive it could be. We can adjust this minor detail with a follow on issue if need be.

          Show
          Andrew Purtell added a comment - Pushed to 0.98, branch-1, and master. I also tested the new commands by creating three snapshots, listing them, then deleting them in one go with delete_table_snapshots. It's a bit odd that the delete_table_snapshots command asks for confirmation where others do not. That is reasonable given how destructive it could be. We can adjust this minor detail with a follow on issue if need be.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #6653 (See https://builds.apache.org/job/HBase-TRUNK/6653/)
          HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev e6bd0c8c155eaf08bb6fe65932fad79fe345c88c)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
          • hbase-shell/src/main/ruby/hbase/admin.rb
          • hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
          • hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          • hbase-shell/src/main/ruby/shell.rb
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #6653 (See https://builds.apache.org/job/HBase-TRUNK/6653/ ) HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev e6bd0c8c155eaf08bb6fe65932fad79fe345c88c) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java hbase-shell/src/main/ruby/hbase/admin.rb hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb hbase-shell/src/main/ruby/shell.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Hide
          Ashish Singhi added a comment -

          Thanks Andrew Purtell.

          We're missing shell tests for the new list_table_snapshots and delete_table_snapshots shell commands

          Actually we are missing shell tests for the complete snapshot feature. I will try to soon create a jira and address this.

          It's a bit odd that the delete_table_snapshots command asks for confirmation where others do not. That is reasonable given how destructive it could be. We can adjust this minor detail with a follow on issue if need be

          AFAIK all the commands which perform operation on a list of a entity asks for a confirmations. For eg: drop_all, delete_all_snapshot e.t.c

          Show
          Ashish Singhi added a comment - Thanks Andrew Purtell . We're missing shell tests for the new list_table_snapshots and delete_table_snapshots shell commands Actually we are missing shell tests for the complete snapshot feature. I will try to soon create a jira and address this. It's a bit odd that the delete_table_snapshots command asks for confirmation where others do not. That is reasonable given how destructive it could be. We can adjust this minor detail with a follow on issue if need be AFAIK all the commands which perform operation on a list of a entity asks for a confirmations. For eg: drop_all, delete_all_snapshot e.t.c
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-1.3 #59 (See https://builds.apache.org/job/HBase-1.3/59/)
          HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev 789d2a94b7e8c2d97c1b52f4f1f0d47922b711a2)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
          • hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          • hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb
          • hbase-shell/src/main/ruby/hbase/admin.rb
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          • hbase-shell/src/main/ruby/shell.rb
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-1.3 #59 (See https://builds.apache.org/job/HBase-1.3/59/ ) HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev 789d2a94b7e8c2d97c1b52f4f1f0d47922b711a2) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb hbase-shell/src/main/ruby/hbase/admin.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java hbase-shell/src/main/ruby/shell.rb
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-0.98 #1059 (See https://builds.apache.org/job/HBase-0.98/1059/)
          HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev f15ad3da641785ae6105f9100d486dc270aa8dcd)

          • hbase-shell/src/main/ruby/hbase/admin.rb
          • hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          • hbase-shell/src/main/ruby/shell.rb
          • hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb
          • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #1059 (See https://builds.apache.org/job/HBase-0.98/1059/ ) HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev f15ad3da641785ae6105f9100d486dc270aa8dcd) hbase-shell/src/main/ruby/hbase/admin.rb hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb hbase-shell/src/main/ruby/shell.rb hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1012 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1012/)
          HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev f15ad3da641785ae6105f9100d486dc270aa8dcd)

          • hbase-shell/src/main/ruby/hbase/admin.rb
          • hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb
          • hbase-shell/src/main/ruby/shell.rb
          • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          • hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1012 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1012/ ) HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev f15ad3da641785ae6105f9100d486dc270aa8dcd) hbase-shell/src/main/ruby/hbase/admin.rb hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb hbase-shell/src/main/ruby/shell.rb hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-1.3-IT #44 (See https://builds.apache.org/job/HBase-1.3-IT/44/)
          HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev 789d2a94b7e8c2d97c1b52f4f1f0d47922b711a2)

          • hbase-shell/src/main/ruby/shell.rb
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          • hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb
          • hbase-shell/src/main/ruby/hbase/admin.rb
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
          • hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-1.3-IT #44 (See https://builds.apache.org/job/HBase-1.3-IT/44/ ) HBASE-8642 [Snapshot] List and delete snapshot by table (apurtell: rev 789d2a94b7e8c2d97c1b52f4f1f0d47922b711a2) hbase-shell/src/main/ruby/shell.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb hbase-shell/src/main/ruby/hbase/admin.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb

            People

            • Assignee:
              Ashish Singhi
              Reporter:
              Julian Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development