Hive
  1. Hive
  2. HIVE-479

Change RCFileOutputFormat's generic signatute to be <WritableComparable, ..>

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      HIVE-479. Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)
      Show
      HIVE-479 . Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)

      Description

      In one test for hive-477, i found that the below command can not sucessfully pass:

      $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0

      It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        5m 50s 1 He Yongqiang 12/May/09 06:25
        Patch Available Patch Available Resolved Resolved
        22d 16h 7m 1 Zheng Shao 03/Jun/09 22:33
        Resolved Resolved Closed Closed
        926d 2h 33m 1 Carl Steinbach 17/Dec/11 00:07
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Zheng Shao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Release Note HIVE-479. Change RCFileOutputFormat's generic signatute. (Yongqiang He via zshao)
        Fix Version/s 0.4.0 [ 12313714 ]
        Resolution Fixed [ 1 ]
        Hide
        Zheng Shao added a comment -

        Committed. Thanks Yongqiang.

        Show
        Zheng Shao added a comment - Committed. Thanks Yongqiang.
        Ashish Thusoo made changes -
        Assignee He Yongqiang [ he yongqiang ]
        He Yongqiang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        He Yongqiang made changes -
        Attachment hive-479-2009-5-12.patch [ 12407844 ]
        He Yongqiang made changes -
        Field Original Value New Value
        Description In one test for hive-477, i found that the below command can not sucessfully pass:
        {noformat}
        $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0
        {noformat}

        It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.
        In one test for hive-477, i found that the below command can not sucessfully pass:

        $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-0.19.0-streaming.jar -input /user/hive/warehouse/tablerc1 -output testHiveWriter -inputformat org.apache.hadoop.hive.ql.io.RCFileInputFormat -outputformat org.apache.hadoop.hive.ql.io.RCFileOutputFormat -mapper org.apache.hadoop.mapred.lib.IdentityMapper -jobconf mapred.work.output.dir=. -jobconf hive.io.rcfile.column.number.conf=32 -jobconf mapred.output.compress=true -numReduceTasks 0

        It is caused by the assumption that RCFileOutputFormat thinks the key's class of passes in records is Void.
        He Yongqiang created issue -

          People

          • Assignee:
            He Yongqiang
            Reporter:
            He Yongqiang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development