Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15806

An endpoint-based export tool

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • Coprocessors, tooling
    • None
    • Reviewed
    • Hide
      org.apache.hadoop.hbase.coprocessor.Export
      Instructs HBase to dump the contents of table to HDFS in a sequence file
      + replaces MR by endpoint (see org.apache.hadoop.hbase.mapreduce.Export)
      + no large data to be transfered between hbase server and client
      + same command line as org.apache.hadoop.hbase.mapreduce.Export
      - user needs to alter table for deploying ExportEndpoint
      - user needs to adjust the endpoint timeout for dumping large data
      - user needs to get the EXECUTE permission
      Show
      org.apache.hadoop.hbase.coprocessor.Export Instructs HBase to dump the contents of table to HDFS in a sequence file + replaces MR by endpoint (see org.apache.hadoop.hbase.mapreduce.Export) + no large data to be transfered between hbase server and client + same command line as org.apache.hadoop.hbase.mapreduce.Export - user needs to alter table for deploying ExportEndpoint - user needs to adjust the endpoint timeout for dumping large data - user needs to get the EXECUTE permission

    Description

      The time for exporting table can be reduced, if we use the endpoint technique to export the hdfs files by the region server rather than by hbase client.

      In my experiments, the elapsed time of endpoint-based export can be less than half of current export tool (enable the hdfs compression)

      But the shortcomings is we need to alter table for deploying the endpoint

      any comments about this? thanks

      Attachments

        1. Experiment.png
          128 kB
          Chia-Ping Tsai
        2. HBASE-15806.patch
          119 kB
          Chia-Ping Tsai
        3. HBASE-15806-v1.patch
          152 kB
          Chia-Ping Tsai
        4. HBASE-15806-v2.patch
          152 kB
          Chia-Ping Tsai
        5. HBASE-15806-v3.patch
          152 kB
          Chia-Ping Tsai
        6. HBASE-15806.v4.patch
          167 kB
          Chia-Ping Tsai
        7. HBASE-15806.v5.patch
          167 kB
          Chia-Ping Tsai
        8. HBASE-15806.v6.patch
          165 kB
          Chia-Ping Tsai
        9. HBASE-15806.v7.patch
          165 kB
          Chia-Ping Tsai
        10. HBASE-15806.v8.patch
          87 kB
          Chia-Ping Tsai
        11. HBASE-15806.v9.patch
          88 kB
          Chia-Ping Tsai
        12. HBASE-15806.v10.patch
          87 kB
          Chia-Ping Tsai
        13. HBASE-15806.v10.patch
          87 kB
          Chia-Ping Tsai
        14. HBASE-15806.v11.patch
          87 kB
          Chia-Ping Tsai

        Issue Links

          Activity

            People

              chia7712 Chia-Ping Tsai
              chia7712 Chia-Ping Tsai
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: