Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3269

Jobsummary logs not being moved to a separate file

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      The jobsummary logs are not being moved to a separate file. Below is the configuration in log4j.properties:

      mapred.jobsummary.logger=INFO,console
      log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
      log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
      log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
      log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
      log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
      log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
      log4j.appender.JSA.DatePattern=.yyyy-MM-dd
      
      1. MAPREDUCE-3269.patch
        1 kB
        Mahadev konar
      2. MAPREDUCE-3269.patch
        1 kB
        Mahadev konar

        Activity

        Ramya Sunil created issue -
        Hide
        Mahadev konar added a comment -

        Minor fixes to set default to INFO, JSA in yarn-daemon.sh. Doing some testing.

        Show
        Mahadev konar added a comment - Minor fixes to set default to INFO, JSA in yarn-daemon.sh. Doing some testing.
        Mahadev konar made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-3269.patch [ 12500772 ]
        Mahadev konar made changes -
        Assignee Mahadev konar [ mahadev ]
        Hide
        Mahadev konar added a comment -

        Tested the fix. This works.

        Show
        Mahadev konar added a comment - Tested the fix. This works.
        Mahadev konar made changes -
        Attachment MAPREDUCE-3269.patch [ 12500777 ]
        Mahadev konar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12500777/MAPREDUCE-3269.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated 8 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500777/MAPREDUCE-3269.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 8 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks Mahadev!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks Mahadev!
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #51 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/51/)
        Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #51 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/51/ ) Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1153 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1153/)
        MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1153 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1153/ ) MAPREDUCE-3269 . Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1231 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1231/)
        MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1231 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1231/ ) MAPREDUCE-3269 . Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #50 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/50/)
        Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #50 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/50/ ) Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1167 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1167/)
        MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1167 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1167/ ) MAPREDUCE-3269 . Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #50 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/50/)
        Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #50 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/50/ ) Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #844 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/844/)
        MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #844 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/844/ ) MAPREDUCE-3269 . Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #872 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/872/)
        MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #872 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/872/ ) MAPREDUCE-3269 . Fixed log4j properties to correctly set logging options for JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #63 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/63/)
        Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #63 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/63/ ) Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #51 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/51/)
        Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #51 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/51/ ) Merge -c 1188999 from trunk to branch-0.23 to complete fix for MAPREDUCE-3269 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        2h 7m 1 Mahadev konar 26/Oct/11 00:15
        Patch Available Patch Available Resolved Resolved
        3h 1 Arun C Murthy 26/Oct/11 03:15
        Resolved Resolved Closed Closed
        19d 22h 33m 1 Arun C Murthy 15/Nov/11 00:49

          People

          • Assignee:
            Mahadev konar
            Reporter:
            Ramya Sunil
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development