Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15806

An endpoint-based export tool

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: Coprocessors, tooling
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      org.apache.hadoop.hbase.coprocessor.Export
      Instructs HBase to dump the contents of table to HDFS in a sequence file
      + replaces MR by endpoint (see org.apache.hadoop.hbase.mapreduce.Export)
      + no large data to be transfered between hbase server and client
      + same command line as org.apache.hadoop.hbase.mapreduce.Export
      - user needs to alter table for deploying ExportEndpoint
      - user needs to adjust the endpoint timeout for dumping large data
      - user needs to get the EXECUTE permission
      Show
      org.apache.hadoop.hbase.coprocessor.Export Instructs HBase to dump the contents of table to HDFS in a sequence file + replaces MR by endpoint (see org.apache.hadoop.hbase.mapreduce.Export) + no large data to be transfered between hbase server and client + same command line as org.apache.hadoop.hbase.mapreduce.Export - user needs to alter table for deploying ExportEndpoint - user needs to adjust the endpoint timeout for dumping large data - user needs to get the EXECUTE permission

      Description

      The time for exporting table can be reduced, if we use the endpoint technique to export the hdfs files by the region server rather than by hbase client.

      In my experiments, the elapsed time of endpoint-based export can be less than half of current export tool (enable the hdfs compression)

      But the shortcomings is we need to alter table for deploying the endpoint

      any comments about this? thanks

        Attachments

        1. Experiment.png
          128 kB
          Chia-Ping Tsai
        2. HBASE-15806.patch
          119 kB
          Chia-Ping Tsai
        3. HBASE-15806.v10.patch
          87 kB
          Chia-Ping Tsai
        4. HBASE-15806.v10.patch
          87 kB
          Chia-Ping Tsai
        5. HBASE-15806.v11.patch
          87 kB
          Chia-Ping Tsai
        6. HBASE-15806.v4.patch
          167 kB
          Chia-Ping Tsai
        7. HBASE-15806.v5.patch
          167 kB
          Chia-Ping Tsai
        8. HBASE-15806.v6.patch
          165 kB
          Chia-Ping Tsai
        9. HBASE-15806.v7.patch
          165 kB
          Chia-Ping Tsai
        10. HBASE-15806.v8.patch
          87 kB
          Chia-Ping Tsai
        11. HBASE-15806.v9.patch
          88 kB
          Chia-Ping Tsai
        12. HBASE-15806-v1.patch
          152 kB
          Chia-Ping Tsai
        13. HBASE-15806-v2.patch
          152 kB
          Chia-Ping Tsai
        14. HBASE-15806-v3.patch
          152 kB
          Chia-Ping Tsai

          Issue Links

            Activity

              People

              • Assignee:
                chia7712 Chia-Ping Tsai
                Reporter:
                chia7712 Chia-Ping Tsai
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: