Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, 2.0.0
    • Component/s: documentation, snapshots
    • Labels:
      None
    1. HBASE-11736.patch
      2 kB
      Misty Stanley-Jones
    2. HBASE-11736.patch
      1 kB
      Misty Stanley-Jones
    3. HBASE-11736.patch
      1 kB
      Misty Stanley-Jones
    4. HBASE-11736.patch
      1 kB
      Misty Stanley-Jones

      Activity

      Misty Stanley-Jones created issue -
      Misty Stanley-Jones made changes -
      Field Original Value New Value
      Fix Version/s 0.99.0 [ 12325675 ]
      Fix Version/s 0.94.21 [ 12326794 ]
      Fix Version/s 0.98.4 [ 12326810 ]
      Misty Stanley-Jones made changes -
      Attachment HBASE-11736.patch [ 12661613 ]
      Hide
      Misty Stanley-Jones added a comment -

      Ready for review.

      Show
      Misty Stanley-Jones added a comment - Ready for review.
      Misty Stanley-Jones made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12661613/HBASE-11736.patch
      against trunk revision .
      ATTACHMENT ID: 12661613

      +1 @author. The patch does not contain any @author tags.

      +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 lineLengths. The patch does not introduce lines longer than 100

      +1 site. The mvn site goal succeeds with this patch.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.master.TestMasterFailover
      org.apache.hadoop.hbase.replication.TestPerTableCFReplication
      org.apache.hadoop.hbase.client.TestReplicaWithCluster

      -1 core zombie tests. There are 1 zombie test(s):

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12661613/HBASE-11736.patch against trunk revision . ATTACHMENT ID: 12661613 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.replication.TestPerTableCFReplication org.apache.hadoop.hbase.client.TestReplicaWithCluster -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10423//console This message is automatically generated.
      Hide
      Matteo Bertozzi added a comment -

      The default behavior is to require all regions of a table to coordinate in order to
      prevent writing to the table, and to perform a flush at the same time. This causes the
      table to be unavailable for writes while the snapshot is being taken.

      There is no locking anywhere in snapshots.
      Coordination is just the process of asking to each Region to perform the snapshot operation and then report back once they are done.
      The default behavior of snapshot is flushing (not locking), which means that the data in the memstore at the moment of the snapshot operation on the region will be included in the snapshot. but you still can write or read while the snapshot operation is going on.

      However, if your set-up can tolerate the possibility of some data not being captured by the snapshot, you can use the <option>SKIP_FLUSH</option> option of the snapshot command to disable locking and flushing while taking the snapshot.

      sort of ok, since there is no locking you don't really know which data is present in the snapshot. "A snapshot is an row-consistent image of what was in the table between the time you asked for the snapshot and the completion". Skip flush will avoid to flush the memstore which means the data not flushed yet will not be included in the snapshot (you don't have any way to know what).
      just emphasize that the only difference is the memstore not flushed

      hbase> snapshot 'mytable', 'snapshot123', 'SKIP_FLUSH'

      That does not work, the correct one is:

      snapshot 'namespace:sourceTable', 'snapshotName', {SKIP_FLUSH => true}

      Using this option can result in data lost if you try to restore a snapshot which is missing data that was written while the snapshot was taken.

      can you reword this avoiding the "can result in data lost". You don't lose any data on restore.
      Since there is no locking between the start and the end of the snapshot operation, you can still insert data while the snapshot is in progress. The snapshot is just a "snapshot at some point in time", so the data inserted on a particular region after the snapshot was taken is not in the snapshot. With the skip flush not only the data inserted after the snapshot operation, but all the data in the memstore is not available.

      Show
      Matteo Bertozzi added a comment - The default behavior is to require all regions of a table to coordinate in order to prevent writing to the table, and to perform a flush at the same time. This causes the table to be unavailable for writes while the snapshot is being taken. There is no locking anywhere in snapshots. Coordination is just the process of asking to each Region to perform the snapshot operation and then report back once they are done. The default behavior of snapshot is flushing (not locking), which means that the data in the memstore at the moment of the snapshot operation on the region will be included in the snapshot. but you still can write or read while the snapshot operation is going on. However, if your set-up can tolerate the possibility of some data not being captured by the snapshot, you can use the <option>SKIP_FLUSH</option> option of the snapshot command to disable locking and flushing while taking the snapshot. sort of ok, since there is no locking you don't really know which data is present in the snapshot. "A snapshot is an row-consistent image of what was in the table between the time you asked for the snapshot and the completion". Skip flush will avoid to flush the memstore which means the data not flushed yet will not be included in the snapshot (you don't have any way to know what). just emphasize that the only difference is the memstore not flushed hbase> snapshot 'mytable', 'snapshot123', 'SKIP_FLUSH' That does not work, the correct one is: snapshot 'namespace:sourceTable', 'snapshotName', {SKIP_FLUSH => true } Using this option can result in data lost if you try to restore a snapshot which is missing data that was written while the snapshot was taken. can you reword this avoiding the "can result in data lost". You don't lose any data on restore. Since there is no locking between the start and the end of the snapshot operation, you can still insert data while the snapshot is in progress. The snapshot is just a "snapshot at some point in time", so the data inserted on a particular region after the snapshot was taken is not in the snapshot. With the skip flush not only the data inserted after the snapshot operation, but all the data in the memstore is not available.
      Hide
      Misty Stanley-Jones added a comment -

      Thanks Matteo Bertozzi, let me know if this is more accurate.

      Show
      Misty Stanley-Jones added a comment - Thanks Matteo Bertozzi , let me know if this is more accurate.
      Misty Stanley-Jones made changes -
      Attachment HBASE-11736.patch [ 12661946 ]
      Hide
      Hadoop QA added a comment -

      +1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12661946/HBASE-11736.patch
      against trunk revision .
      ATTACHMENT ID: 12661946

      +1 @author. The patch does not contain any @author tags.

      +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 lineLengths. The patch does not introduce lines longer than 100

      +1 site. The mvn site goal succeeds with this patch.

      +1 core tests. The patch passed unit tests in .

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12661946/HBASE-11736.patch against trunk revision . ATTACHMENT ID: 12661946 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10445//console This message is automatically generated.
      Hide
      Matteo Bertozzi added a comment -

      almost, but we never lock

      The default behavior is to require all regions of a table to coordinate in order to prevent writing to the table, and to perform a flush of data in memory at the same time

      we don't lock writes, we just do a flush. and it is basically the only difference between SKIP_FLUSH = false and SKIP_FLUSH = true.

      Using this option means that data can be inserted into a table while a snapshot is in progress, and that new data will not be included in the snapshot.

      the first part seems to refer to the "write locked" on the other case, which is not true.
      maybe just something like "the data present in the memstore will not be included in the snapshot"?

      There is no way to determine the newest data that will be included in the snapshot if flushing is disabled.

      This is true in both case since there is no lock around the snapshot operation.
      Let me try to do an example:

      • Create table
      • Add data to table (RS-1 memstore has: [A, B, C], RS-2 memstore has: [M, N, O])
      • assuming that you don't do any write now, and you take a snapshot
        • The SKIP_FLUSH = false will result in an empty snapshot, since all the data is in the memstore
        • The SKIP_FLUSH = true will result in a snapshot with [A, B, C] and [M, N, O] since you flush the memstores
          now, if you have writes coming in.. but keep in mind there is no lock
          The RS-1 may start flushing/taking the snapshot before any write so you get [A, B, C] and concurrent writes will be added to the "new memstore".
          RS-2 may start flushing/taking the snapshot after some writes so you get [M, N, O, P, Q] but concurrent writes during the flush will be added to the new memstore
      Show
      Matteo Bertozzi added a comment - almost, but we never lock The default behavior is to require all regions of a table to coordinate in order to prevent writing to the table, and to perform a flush of data in memory at the same time we don't lock writes, we just do a flush. and it is basically the only difference between SKIP_FLUSH = false and SKIP_FLUSH = true. Using this option means that data can be inserted into a table while a snapshot is in progress, and that new data will not be included in the snapshot. the first part seems to refer to the "write locked" on the other case, which is not true. maybe just something like "the data present in the memstore will not be included in the snapshot"? There is no way to determine the newest data that will be included in the snapshot if flushing is disabled. This is true in both case since there is no lock around the snapshot operation. Let me try to do an example: Create table Add data to table (RS-1 memstore has: [A, B, C] , RS-2 memstore has: [M, N, O] ) assuming that you don't do any write now, and you take a snapshot The SKIP_FLUSH = false will result in an empty snapshot, since all the data is in the memstore The SKIP_FLUSH = true will result in a snapshot with [A, B, C] and [M, N, O] since you flush the memstores now, if you have writes coming in.. but keep in mind there is no lock The RS-1 may start flushing/taking the snapshot before any write so you get [A, B, C] and concurrent writes will be added to the "new memstore". RS-2 may start flushing/taking the snapshot after some writes so you get [M, N, O, P, Q] but concurrent writes during the flush will be added to the new memstore
      Hide
      Misty Stanley-Jones added a comment -

      Rewrite to remove implied behavior relating to locking, and to add some clarity. Let me know if this works.

      Show
      Misty Stanley-Jones added a comment - Rewrite to remove implied behavior relating to locking, and to add some clarity. Let me know if this works.
      Misty Stanley-Jones made changes -
      Attachment HBASE-11736.patch [ 12662644 ]
      Hide
      Matteo Bertozzi added a comment -

      I'm +1 with it. There are a couple of words that may not be 100% correct but I think is correct enough to give the idea. I'll wait another day before committing to see if you or someone else have alternatives.

      whether a very recent insert or update

      "very recent" is probably not the right way to describe it.. maybe "concurrent" is better.

      A snapshot is only a representation of a table at a given point in time

      some one can argue on the meaning of "at a given point in time" since is more "a window", since the time it takes to reach every server can take from few seconds up to a minute depending on how slow is your machine.

      Show
      Matteo Bertozzi added a comment - I'm +1 with it. There are a couple of words that may not be 100% correct but I think is correct enough to give the idea. I'll wait another day before committing to see if you or someone else have alternatives. whether a very recent insert or update "very recent" is probably not the right way to describe it.. maybe "concurrent" is better. A snapshot is only a representation of a table at a given point in time some one can argue on the meaning of "at a given point in time" since is more "a window", since the time it takes to reach every server can take from few seconds up to a minute depending on how slow is your machine.
      Hide
      Misty Stanley-Jones added a comment -

      Let's see if this is better, Matteo Bertozzi

      Show
      Misty Stanley-Jones added a comment - Let's see if this is better, Matteo Bertozzi
      Misty Stanley-Jones made changes -
      Attachment HBASE-11736.patch [ 12662942 ]
      Hide
      Matteo Bertozzi added a comment -

      +1

      Show
      Matteo Bertozzi added a comment - +1
      Matteo Bertozzi made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Fix Version/s 2.0.0 [ 12327188 ]
      Fix Version/s 0.94.21 [ 12326794 ]
      Fix Version/s 0.98.4 [ 12326810 ]
      Resolution Fixed [ 1 ]
      Hide
      Hudson added a comment -

      FAILURE: Integrated in HBase-TRUNK #5414 (See https://builds.apache.org/job/HBase-TRUNK/5414/)
      HBASE-11736 Document SKIP_FLUSH snapshot option (Misty Stanley-Jones) (matteo.bertozzi: rev dd3c9da27a4864ade4e3ac2209d081d42353ccf2)

      • src/main/docbkx/ops_mgt.xml
      Show
      Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5414 (See https://builds.apache.org/job/HBase-TRUNK/5414/ ) HBASE-11736 Document SKIP_FLUSH snapshot option (Misty Stanley-Jones) (matteo.bertozzi: rev dd3c9da27a4864ade4e3ac2209d081d42353ccf2) src/main/docbkx/ops_mgt.xml
      Hide
      Hudson added a comment -

      FAILURE: Integrated in HBase-1.0 #115 (See https://builds.apache.org/job/HBase-1.0/115/)
      HBASE-11736 Document SKIP_FLUSH snapshot option (Misty Stanley-Jones) (matteo.bertozzi: rev aee3990eb50ce526e7b90da371b2e522abb802b4)

      • src/main/docbkx/ops_mgt.xml
      Show
      Hudson added a comment - FAILURE: Integrated in HBase-1.0 #115 (See https://builds.apache.org/job/HBase-1.0/115/ ) HBASE-11736 Document SKIP_FLUSH snapshot option (Misty Stanley-Jones) (matteo.bertozzi: rev aee3990eb50ce526e7b90da371b2e522abb802b4) src/main/docbkx/ops_mgt.xml
      Hide
      Enis Soztutar added a comment -

      Closing this issue after 0.99.0 release.

      Show
      Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
      Enis Soztutar made changes -
      Status Resolved [ 5 ] Closed [ 6 ]

        People

        • Assignee:
          Misty Stanley-Jones
          Reporter:
          Misty Stanley-Jones
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development