Hive
  1. Hive
  2. HIVE-479

Change RCFileOutputFormat's generic signatute to be <WritableComparable, ..>

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      HIVE-479. Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)
      Show
      HIVE-479 . Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)

      Description

      In one test for hive-477, i found that the below command can not sucessfully pass:

      $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0

      It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.

        Activity

        He Yongqiang created issue -
        He Yongqiang made changes -
        Field Original Value New Value
        Description In one test for hive-477, i found that the below command can not sucessfully pass:
        {noformat}
        $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0
        {noformat}

        It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.
        In one test for hive-477, i found that the below command can not sucessfully pass:

        $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0

        It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.
        He Yongqiang made changes -
        Attachment hive-479-2009-5-12.patch [ 12407844 ]
        He Yongqiang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ashish Thusoo made changes -
        Assignee He Yongqiang [ he yongqiang ]
        Zheng Shao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Release Note HIVE-479. Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)
        Fix Version/s 0.4.0 [ 12313714 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            He Yongqiang
            Reporter:
            He Yongqiang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development