HBase
  1. HBase
  2. HBASE-8202

MultiTableOutputFormat should support writing to another HBase cluster

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: mapreduce
    • Labels:
      None

      Description

      This was brought up by David Koch in thread 'hbase.mapred.output.quorum ignored in Mapper job with HDFS source and HBase sink' where he wanted to import a file on HDFS from one cluster A (source) into HBase
      tables on a different cluster B (destination) using a Mapper job with an
      HBase sink.

      Here is my analysis:

      MultiTableOutputFormat doesn't extend TableOutputFormat:

      public class MultiTableOutputFormat extends OutputFormat<ImmutableBytesWritable, Mutation> {
      

      Relevant configuration w.r.t. output quorum is setup in TableOutputFormat#setConf():

        public void setConf(Configuration otherConf) {
          this.conf = HBaseConfiguration.create(otherConf);
          String tableName = this.conf.get(OUTPUT_TABLE);
          if(tableName == null || tableName.length() <= 0) {
            throw new IllegalArgumentException("Must specify table name");
          }
          String address = this.conf.get(QUORUM_ADDRESS);
          int zkClientPort = conf.getInt(QUORUM_PORT, 0);
          String serverClass = this.conf.get(REGION_SERVER_CLASS);
          String serverImpl = this.conf.get(REGION_SERVER_IMPL);
          try {
            if (address != null) {
              ZKUtil.applyClusterKeyToConf(this.conf, address);
            }
      

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Ted Yu
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development