Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      I have been running wordcount out of the 23 examples jar. It says it succeeds but doesn't actually output a file.

      hadoop jar examples/hadoop-mapreduce-0.23.0-SNAPSHOT/hadoop-mapreduce-examples-0.23.0-SNAPSHOT.jar wordcount input output2

      input file is really basic:
      fdksajl
      dlkfsajlfljda;j
      kldfsjallj
      test
      one
      two
      test

        Activity

        Thomas Graves created issue -
        Mahadev konar made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Critical [ 2 ]
        Hide
        Siddharth Seth added a comment -

        Saw the same thing. Looks like reduce tasks are not being run.
        Setting the number of reduce tasks (-Dmapred.reduce.tasks=X) works though.

        Show
        Siddharth Seth added a comment - Saw the same thing. Looks like reduce tasks are not being run. Setting the number of reduce tasks (-Dmapred.reduce.tasks=X) works though.
        Thomas Graves made changes -
        Assignee Thomas Graves [ tgraves ]
        Hide
        Thomas Graves added a comment -

        It looks like the mapred-defaults.xml isn't included in any of the mapreduce-client jar files. It used to be in the client-core jar. That is what was setting the default number of reduces to 1. The code current defaults to 0 if not specified.

        job.numReduceTasks = job.conf.getInt(MRJobConfig.NUM_REDUCES, 0);

        Show
        Thomas Graves added a comment - It looks like the mapred-defaults.xml isn't included in any of the mapreduce-client jar files. It used to be in the client-core jar. That is what was setting the default number of reduces to 1. The code current defaults to 0 if not specified. job.numReduceTasks = job.conf.getInt(MRJobConfig.NUM_REDUCES, 0);
        Hide
        Thomas Graves added a comment -

        it looks like the ./mr-client/hadoop-mapreduce-client-core/src/main/resources/ directory never got merged to trunk from mr-279 branch. That directory included the mapred-default.xml that included the default reducers.

        Assigned to Mahadev to investigate if it was intentional or accidentally missed during the merge.

        Show
        Thomas Graves added a comment - it looks like the ./mr-client/hadoop-mapreduce-client-core/src/main/resources/ directory never got merged to trunk from mr-279 branch. That directory included the mapred-default.xml that included the default reducers. Assigned to Mahadev to investigate if it was intentional or accidentally missed during the merge.
        Thomas Graves made changes -
        Summary wordcount succeeds but doesn't output file mapred-default.xml not merged from mr279
        Assignee Thomas Graves [ tgraves ] Mahadev konar [ mahadev ]
        Hide
        Arun C Murthy added a comment -

        Thanks Thomas, I fixed it by doing an 'svn copy' to both trunk and branch-0.23.

        Show
        Arun C Murthy added a comment - Thanks Thomas, I fixed it by doing an 'svn copy' to both trunk and branch-0.23.
        Arun C Murthy made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Mahadev konar [ mahadev ] Arun C Murthy [ acmurthy ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        5h 50m 1 Arun C Murthy 09/Sep/11 03:54
        Resolved Resolved Closed Closed
        66d 21h 56m 1 Arun C Murthy 15/Nov/11 00:50

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development