Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5108 Changes needed for Binary Compatibility for MR applications via YARN
  3. MAPREDUCE-5160

Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find their inner classes when running on Yarn

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 2.1.0-beta
    • Component/s: None
    • Labels:
      None

      Description

      Aggregatewordcount and Aggregatewordhist of hadoop-1 cannot run on hadoop-2 due to org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob.createValueAggregatorJob(String args[], Class<?> caller) is not available on hadoop-2 (see MAPREDUCE-5159).

      After I changed Aggregatewordcount and Aggregatewordhist to use createValueAggregatorJob(String args[], Class<? extends ValueAggregatorDescriptor>[] descriptors), which is available on hadoop-2, the two examples could be accepted and run on Yarn.

      However, the two examples still failed, because their inner classes, WordCountPlugInClass and AggregateWordHistogramPlugin, cannot be found in runtime, respectively. Both the plugin classes extend org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zjshen Zhijie Shen
                Reporter:
                zjshen Zhijie Shen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: