Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7704

migration tool that checks presence of HFile V1 files

    Details

    • Type: Task
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds tool to check if any v1 hfiles present under hbase.rootdir. Run this tool before you move to hbase 0.95/0.96 because hfile v1 files are no longer supported in the newer versions of hbase (If an hfile v1 is found, run a major compaction on the offending, containing region and that should purge the v1 file).

      Here is what the usage for the tool looks like:


      {code}
      bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -h
      usage: HFileV1Detector [-h] [-n <arg>] [-p <arg>]
       -h,--help Help
       -n,--numberOfThreads <arg> Number of threads to use while processing
                                    HFiles.
       -p,--path <arg> Path to a table, or hbase installation
      In case no option is provided, it processes hbase.rootdir using 10 threads.
      Example:
       To detect any HFileV1 in a given hbase installation '/myhbase':
       $ $HBASE_HOME/bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -p /myhbase
      {code}
      Show
      Adds tool to check if any v1 hfiles present under hbase.rootdir. Run this tool before you move to hbase 0.95/0.96 because hfile v1 files are no longer supported in the newer versions of hbase (If an hfile v1 is found, run a major compaction on the offending, containing region and that should purge the v1 file). Here is what the usage for the tool looks like: {code} bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -h usage: HFileV1Detector [-h] [-n <arg>] [-p <arg>]  -h,--help Help  -n,--numberOfThreads <arg> Number of threads to use while processing                               HFiles.  -p,--path <arg> Path to a table, or hbase installation In case no option is provided, it processes hbase.rootdir using 10 threads. Example:  To detect any HFileV1 in a given hbase installation '/myhbase':  $ $HBASE_HOME/bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -p /myhbase {code}

      Description

      Below was Stack's comment from HBASE-7660:

      Regards the migration 'tool', or 'tool' to check for presence of v1 files, I imagine it as an addition to the hfile tool http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a bunch of args including printing out meta. We could add an option to print out version only – or return 1 if version 1 or some such – and then do a bit of code to just list all hfiles and run this script against each. Could MR it if too many files.

        Attachments

        1. HBASE-7704-v6.patch
          12 kB
          Himanshu Vashishtha
        2. HBASE-7704-v5.patch
          12 kB
          Himanshu Vashishtha
        3. HBase-7704-v4.patch
          11 kB
          Himanshu Vashishtha
        4. HBase-7704-v3.patch
          11 kB
          Himanshu Vashishtha
        5. HBase-7704-v2.patch
          11 kB
          Himanshu Vashishtha
        6. HBase-7704-v1.patch
          10 kB
          Himanshu Vashishtha

          Issue Links

            Activity

              People

              • Assignee:
                v.himanshu Himanshu Vashishtha
                Reporter:
                yuzhihong@gmail.com Ted Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: