Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9397

Snapshots with the same name are allowed to proceed concurrently

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.2, 0.94.11
    • Fix Version/s: 0.94.12, 0.96.0
    • Component/s: snapshots
    • Labels:
      None

      Description

      Snapshots with the same name (but on different tables) are allowed to proceed concurrently.
      This seems to be loop hole created by allowing multiple snapshots (on different tables) to run concurrently.
      There are two checks in SnapshotManager, but fail to catch this particular case.
      In isSnapshotCompleted(), we only check the completed snapshot directory.
      In isTakingSnapshot(), we only check for the same table name.

      The end result is the concurrently running snapshots with the same name are overlapping and messing up each other. For example, cleaning up the other's snapshot working directory in .hbase-snapshot/.tmp/snapshot-name.

      2013-08-29 18:25:13,443 ERROR org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler: Failed taking snapshot { ss=mysnapshot table=TestTable type=FLUSH } due to exception:Couldn't read snapshot info from:hdfs://hdtest009:9000/hbase/.hbase-snapshot/.tmp/mysnapshot/.snapshotinfo
      org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read snapshot info from:hdfs://hdtest009:9000/hbase/.hbase-snapshot/.tmp/mysnapshot/.snapshotinfo
              at org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:321)
              at org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshotDescription(MasterSnapshotVerifier.java:123)
      

        Attachments

        1. HBASE-9397-trunk-v2.patch
          3 kB
          Jerry He
        2. HBASE-9397-trunk.patch
          3 kB
          Jerry He
        3. HBASE-9397-0.94-v2.patch
          3 kB
          Jerry He
        4. HBASE-9397-0.94.patch
          3 kB
          Jerry He

          Activity

            People

            • Assignee:
              jinghe Jerry He
              Reporter:
              jinghe Jerry He
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: