Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6271

org.apache.hadoop.mapreduce.Cluster GetJob() display warn log

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: client
    • Labels:

      Description

      When using getJob() with MapReduce 2.7, warn log caused by configuration loaded twice is displayed every time. And when job completed, this function will display warn log of "java.io.FileNotFoundException"

      And I think this is related with MAPREDUCE-5875, the change in GetJob() seems to be not needed, cause it's only for test.

      15/03/04 13:41:23 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:23 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:24 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:24 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:25 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:25 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:26 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:26 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:27 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:27 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:28 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:28 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:29 WARN conf.Configuration: hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:29 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.37 sec
      15/03/04 13:41:30 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:30 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:31 WARN mapreduce.Cluster: Job conf missing on clusterjava.io.FileNotFoundException: File does not exist: /yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml
      

        Attachments

        1. MAPREDUCE-6271.v2.patch
          1 kB
          Peng Zhang
        2. MR-6271.patch
          3 kB
          Peng Zhang

          Issue Links

            Activity

              People

              • Assignee:
                peng.zhang Peng Zhang
                Reporter:
                peng.zhang Peng Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: