Hadoop Common
  1. Hadoop Common
  2. HADOOP-7715

see log4j Error when running mr jobs and certain dfs calls

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: conf
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Removed unnecessary security logger configuration. (Eric Yang)
    1. HADOOP-7715.patch
      0.6 kB
      Eric Yang
    2. HADOOP-7715-trunk.patch
      0.7 kB
      Eric Yang

      Activity

      Arpit Gupta created issue -
      Hide
      Arpit Gupta added a comment -

      While running MR jobs such as teragen, terasort etc and dfs calls such as fetchdt i see the following printed to the console

      log4j:ERROR Attempted to append to closed appender named [console].
      log4j:ERROR Attempted to append to closed appender named [console].
      log4j:ERROR Attempted to append to closed appender named [console].
      log4j:ERROR Attempted to append to closed appender named [console].
      log4j:ERROR Attempted to append to closed appender named [console].
      log4j:ERROR Attempted to append to closed appender named [console].

      We need to investigate why this is happening.

      Show
      Arpit Gupta added a comment - While running MR jobs such as teragen, terasort etc and dfs calls such as fetchdt i see the following printed to the console log4j:ERROR Attempted to append to closed appender named [console] . log4j:ERROR Attempted to append to closed appender named [console] . log4j:ERROR Attempted to append to closed appender named [console] . log4j:ERROR Attempted to append to closed appender named [console] . log4j:ERROR Attempted to append to closed appender named [console] . log4j:ERROR Attempted to append to closed appender named [console] . We need to investigate why this is happening.
      Hide
      Arpit Gupta added a comment -

      Eric could you look into this?

      Show
      Arpit Gupta added a comment - Eric could you look into this?
      Arpit Gupta made changes -
      Field Original Value New Value
      Assignee Eric Yang [ eyang ]
      Hide
      Eric Yang added a comment -

      The recent addition in HADOOP-7681 contains:

      log4j.category.SecurityLogger=${security.audit.logger}
      

      This property should not exist in log4j.properties template.

      Show
      Eric Yang added a comment - The recent addition in HADOOP-7681 contains: log4j.category.SecurityLogger=${security.audit.logger} This property should not exist in log4j.properties template.
      Hide
      Eric Yang added a comment -

      Removed unnecessary security logger configuration for branch-20-security.

      Show
      Eric Yang added a comment - Removed unnecessary security logger configuration for branch-20-security.
      Eric Yang made changes -
      Attachment HADOOP-7715.patch [ 12497543 ]
      Hide
      Eric Yang added a comment -

      Same patch for trunk.

      Show
      Eric Yang added a comment - Same patch for trunk.
      Eric Yang made changes -
      Attachment HADOOP-7715-trunk.patch [ 12497544 ]
      Eric Yang made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Hide
      Arpit Gupta added a comment -

      +1 tested the change on deployed system

      Show
      Arpit Gupta added a comment - +1 tested the change on deployed system
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12497544/HADOOP-7715-trunk.patch
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      -1 tests included. The patch doesn't appear to include any new or modified tests.
      Please justify why no new tests are needed for this patch.
      Also please list what manual steps were performed to verify this patch.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 core tests. The patch passed unit tests in .

      +1 contrib tests. The patch passed contrib unit tests.

      Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/257//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/257//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497544/HADOOP-7715-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/257//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/257//console This message is automatically generated.
      Hide
      Eric Yang added a comment -

      I just committed this to 0.20 security branch, 0.20.205, 0.23, and trunk. Thanks Arpit.

      Show
      Eric Yang added a comment - I just committed this to 0.20 security branch, 0.20.205, 0.23, and trunk. Thanks Arpit.
      Eric Yang made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Hadoop Flags Reviewed [ 10343 ]
      Release Note Removed unnecessary security logger configuration. (Eric Yang)
      Resolution Fixed [ 1 ]
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Common-trunk-Commit #1007 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1007/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1007 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1007/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk-Commit #1085 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1085/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1085 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1085/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk-Commit #1027 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1027/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1027 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1027/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178665
      Files :

      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #36 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/36/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178665 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #820 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/820/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178665
      Files :

      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #29 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/29/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178665 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/)
      HADOOP-7715. Removed unnecessary security logger configuration. (Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #850 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/850/ ) HADOOP-7715 . Removed unnecessary security logger configuration. (Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178662 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
      Hide
      Matt Foley added a comment -

      corrected Fix Versions field, based on commit dates.

      Show
      Matt Foley added a comment - corrected Fix Versions field, based on commit dates.
      Matt Foley made changes -
      Fix Version/s 0.23.0 [ 12315569 ]
      Fix Version/s 0.20.205.0 [ 12316390 ]
      Fix Version/s 1.0.0 [ 12318244 ]

        People

        • Assignee:
          Eric Yang
          Reporter:
          Arpit Gupta
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development