Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7704

migration tool that checks presence of HFile V1 files

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.95.1
    • None
    • None
    • Reviewed
    • Hide
      Adds tool to check if any v1 hfiles present under hbase.rootdir. Run this tool before you move to hbase 0.95/0.96 because hfile v1 files are no longer supported in the newer versions of hbase (If an hfile v1 is found, run a major compaction on the offending, containing region and that should purge the v1 file).

      Here is what the usage for the tool looks like:


      {code}
      bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -h
      usage: HFileV1Detector [-h] [-n <arg>] [-p <arg>]
       -h,--help Help
       -n,--numberOfThreads <arg> Number of threads to use while processing
                                    HFiles.
       -p,--path <arg> Path to a table, or hbase installation
      In case no option is provided, it processes hbase.rootdir using 10 threads.
      Example:
       To detect any HFileV1 in a given hbase installation '/myhbase':
       $ $HBASE_HOME/bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -p /myhbase
      {code}
      Show
      Adds tool to check if any v1 hfiles present under hbase.rootdir. Run this tool before you move to hbase 0.95/0.96 because hfile v1 files are no longer supported in the newer versions of hbase (If an hfile v1 is found, run a major compaction on the offending, containing region and that should purge the v1 file). Here is what the usage for the tool looks like: {code} bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -h usage: HFileV1Detector [-h] [-n <arg>] [-p <arg>]  -h,--help Help  -n,--numberOfThreads <arg> Number of threads to use while processing                               HFiles.  -p,--path <arg> Path to a table, or hbase installation In case no option is provided, it processes hbase.rootdir using 10 threads. Example:  To detect any HFileV1 in a given hbase installation '/myhbase':  $ $HBASE_HOME/bin/hbase org.apache.hadoop.hbase.util.HFileV1Detector -p /myhbase {code}

    Description

      Below was Stack's comment from HBASE-7660:

      Regards the migration 'tool', or 'tool' to check for presence of v1 files, I imagine it as an addition to the hfile tool http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a bunch of args including printing out meta. We could add an option to print out version only – or return 1 if version 1 or some such – and then do a bit of code to just list all hfiles and run this script against each. Could MR it if too many files.

      Attachments

        1. HBase-7704-v1.patch
          10 kB
          Himanshu Vashishtha
        2. HBase-7704-v2.patch
          11 kB
          Himanshu Vashishtha
        3. HBase-7704-v3.patch
          11 kB
          Himanshu Vashishtha
        4. HBase-7704-v4.patch
          11 kB
          Himanshu Vashishtha
        5. HBASE-7704-v5.patch
          12 kB
          Himanshu Vashishtha
        6. HBASE-7704-v6.patch
          12 kB
          Himanshu Vashishtha

        Issue Links

          Activity

            People

              v.himanshu Himanshu Vashishtha
              yuzhihong@gmail.com Ted Yu
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: