HBase
  1. HBase
  2. HBASE-5547

Don't delete HFiles when in "backup mode"

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key.

      In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:
        <property>
          <name>hbase.master.logcleaner.plugins</name>
          <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>
       </property>

      It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.
      Show
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key. In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:   <property>     <name>hbase.master.logcleaner.plugins</name>     <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>  </property> It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.

      Description

      This came up in a discussion I had with Stack.
      It would be nice if HBase could be notified that a backup is in progress (via a znode for example) and in that case either:
      1. rename HFiles to be delete to <file>.bck
      2. rename the HFiles into a special directory
      3. rename them to a general trash directory (which would not need to be tied to backup mode).

      That way it should be able to get a consistent backup based on HFiles (HDFS snapshots or hard links would be better options here, but we do not have those).

      #1 makes cleanup a bit harder.

      1. java_HBASE-5547_v4.patch
        195 kB
        Jesse Yates
      2. java_HBASE-5547_v5.patch
        195 kB
        Jesse Yates
      3. java_HBASE-5547_v6.patch
        195 kB
        Jesse Yates
      4. java_HBASE-5547_v7.patch
        162 kB
        Jesse Yates
      5. hbase-5447-v8.patch
        168 kB
        Ted Yu
      6. hbase-5447-v8.patch
        168 kB
        Jesse Yates
      7. hbase-5547-v9.patch
        185 kB
        Jesse Yates
      8. 5547-v12.txt
        184 kB
        Ted Yu
      9. java_HBASE-5547_v13.patch
        190 kB
        Jesse Yates
      10. java_HBASE-5547_v14.patch
        194 kB
        Jesse Yates
      11. java_HBASE-5547_v15.patch
        196 kB
        Ted Yu
      12. 5547-v16.txt
        196 kB
        Ted Yu
      13. java_HBASE-5547.addendum
        7 kB
        Jesse Yates
      14. java_HBASE-5547.addendum-v1
        13 kB
        Jesse Yates
      15. java_HBASE-5547.addendum-v2
        7 kB
        Jesse Yates
      16. 5547.addendum-v3
        7 kB
        Ted Yu
      17. 5547-addendum-v4.txt
        0.7 kB
        Ted Yu
      18. hbase-5547-0.94-backport-v0.patch
        147 kB
        Jesse Yates

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Jesse Yates
              Reporter:
              Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development