Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6271

org.apache.hadoop.mapreduce.Cluster GetJob() display warn log

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.7.0
    • None
    • client

    Description

      When using getJob() with MapReduce 2.7, warn log caused by configuration loaded twice is displayed every time. And when job completed, this function will display warn log of "java.io.FileNotFoundException"

      And I think this is related with MAPREDUCE-5875, the change in GetJob() seems to be not needed, cause it's only for test.

      15/03/04 13:41:23 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:23 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:24 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:24 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:25 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:25 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:26 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:26 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:27 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:27 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:28 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:28 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:29 WARN conf.Configuration: hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:29 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.37 sec
      15/03/04 13:41:30 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
      15/03/04 13:41:30 WARN conf.Configuration: hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
      15/03/04 13:41:31 WARN mapreduce.Cluster: Job conf missing on clusterjava.io.FileNotFoundException: File does not exist: /yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml
      

      Attachments

        1. MR-6271.patch
          3 kB
          Peng Zhang
        2. MAPREDUCE-6271.v2.patch
          1 kB
          Peng Zhang

        Issue Links

          Activity

            People

              peng.zhang Peng Zhang
              peng.zhang Peng Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: