Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5547

Don't delete HFiles when in "backup mode"

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key.

      In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:
        <property>
          <name>hbase.master.logcleaner.plugins</name>
          <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>
       </property>

      It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.
      Show
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key. In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:   <property>     <name>hbase.master.logcleaner.plugins</name>     <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>  </property> It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.

      Description

      This came up in a discussion I had with Stack.
      It would be nice if HBase could be notified that a backup is in progress (via a znode for example) and in that case either:
      1. rename HFiles to be delete to <file>.bck
      2. rename the HFiles into a special directory
      3. rename them to a general trash directory (which would not need to be tied to backup mode).

      That way it should be able to get a consistent backup based on HFiles (HDFS snapshots or hard links would be better options here, but we do not have those).

      #1 makes cleanup a bit harder.

        Attachments

        1. java_HBASE-5547.addendum-v2
          7 kB
          Jesse Yates
        2. java_HBASE-5547.addendum-v1
          13 kB
          Jesse Yates
        3. java_HBASE-5547.addendum
          7 kB
          Jesse Yates
        4. java_HBASE-5547_v7.patch
          162 kB
          Jesse Yates
        5. java_HBASE-5547_v6.patch
          195 kB
          Jesse Yates
        6. java_HBASE-5547_v5.patch
          195 kB
          Jesse Yates
        7. java_HBASE-5547_v4.patch
          195 kB
          Jesse Yates
        8. java_HBASE-5547_v15.patch
          196 kB
          Ted Yu
        9. java_HBASE-5547_v14.patch
          194 kB
          Jesse Yates
        10. java_HBASE-5547_v13.patch
          190 kB
          Jesse Yates
        11. hbase-5547-v9.patch
          185 kB
          Jesse Yates
        12. hbase-5547-0.94-backport-v0.patch
          147 kB
          Jesse Yates
        13. hbase-5447-v8.patch
          168 kB
          Ted Yu
        14. hbase-5447-v8.patch
          168 kB
          Jesse Yates
        15. 5547-v16.txt
          196 kB
          Ted Yu
        16. 5547-v12.txt
          184 kB
          Ted Yu
        17. 5547-addendum-v4.txt
          0.7 kB
          Ted Yu
        18. 5547.addendum-v3
          7 kB
          Ted Yu

          Issue Links

            Activity

              People

              • Assignee:
                jesse_yates Jesse Yates
                Reporter:
                lhofhansl Lars Hofhansl
              • Votes:
                0 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: