Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7186

JobSubmitter#checkSpecs doesn not handle the situation when reduceNums is set less than zero.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: client
    • Labels:
      None

      Description

      JobSubmitter#checkSpecs doesn not judge whether reduceNums is less than zero.

      private void checkSpecs(Job job) throws ClassNotFoundException,
      	InterruptedException, IOException {
      	JobConf jConf = (JobConf)job.getConfiguration();
      	// Check the output specification
      	if (jConf.getNumReduceTasks() == 0 ?
      	jConf.getUseNewMapper() : jConf.getUseNewReducer()) {
      	org.apache.hadoop.mapreduce.OutputFormat<?, ?> output =
      	ReflectionUtils.newInstance(job.getOutputFormatClass(),
      	job.getConfiguration());
      	output.checkOutputSpecs(job);
      	} else {
      	jConf.getOutputFormat().checkOutputSpecs(jtFs, jConf);
      	}
      	}
      	

      The code does handle the situation when reduceTaskNums is set to -1 or any other value less than zero.

      When use hive, the "mapred.reduce.tasks" has a default value -1.

       

        Attachments

        1. screenshot-1.png
          96 kB
          Hongyuan Li
        2. MAPREDUCE-7186-001.patch
          1 kB
          Hongyuan Li
        3. MAPREDUCE-7186-002.patch
          1 kB
          Hongyuan Li

          Issue Links

            Activity

              People

              • Assignee:
                Hongyuan Li Hongyuan Li
                Reporter:
                Hongyuan Li Hongyuan Li
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: