Hadoop Common
  1. Hadoop Common
  2. HADOOP-7681

log4j.properties is missing properties for security audit and hdfs audit should be changed to info

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: conf
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      HADOOP-7681. Fixed security and hdfs audit log4j properties
      (Arpit Gupta via Eric Yang)
      Show
      HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang)

      Description

      log4j.properties defines the security audit and hdfs audit files but is missing properties for security audit which causes security audit logs to not be present and also updates the hdfs audit to log at a WARN level. hdfs-audit logs should be at the INFO level so admin's/users can track when the namespace got the appropriate change.

      1. HADOOP-7681.20s.patch
        4 kB
        Arpit Gupta
      2. HADOOP-7681.patch
        4 kB
        Arpit Gupta
      3. HADOOP-7681-1.patch
        4 kB
        Arpit Gupta
      4. HADOOP-7681-2.patch
        4 kB
        Arpit Gupta

        Activity

        Hide
        Arpit Gupta added a comment -

        security audit log was missing the following

        security.audit.logger=INFO,console
        hadoop.security.log.file=SecurityAuth.audit
        log4j.category.SecurityLogger=$

        {security.audit.logger}

        log4j.appender.DRFAS.DatePattern=.yyyy-MM-dd

        and hdfs audit had the following line which should be removed
        log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=WARN

        Show
        Arpit Gupta added a comment - security audit log was missing the following security.audit.logger=INFO,console hadoop.security.log.file=SecurityAuth.audit log4j.category.SecurityLogger=$ {security.audit.logger} log4j.appender.DRFAS.DatePattern=.yyyy-MM-dd and hdfs audit had the following line which should be removed log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=WARN
        Hide
        Arpit Gupta added a comment -

        patch for trunk.

        moved job summary properties to one place, added missing properties for security audit and removed mr audit properties that were commented out.

        Show
        Arpit Gupta added a comment - patch for trunk. moved job summary properties to one place, added missing properties for security audit and removed mr audit properties that were commented out.
        Hide
        Arpit Gupta added a comment -

        patch for branch 20 security

        added apache license.
        moved job summary properties to one place.
        removed audit loggers being set to WARN.

        Show
        Arpit Gupta added a comment - patch for branch 20 security added apache license. moved job summary properties to one place. removed audit loggers being set to WARN.
        Hide
        Eric Yang added a comment -

        +1 looks good.

        Show
        Eric Yang added a comment - +1 looks good.
        Hide
        Arpit Gupta added a comment -

        forgot to remove some duplicate properties.

        Show
        Arpit Gupta added a comment - forgot to remove some duplicate properties.
        Hide
        Arpit Gupta added a comment -

        re attaching the same patch but with a different name.

        Show
        Arpit Gupta added a comment - re attaching the same patch but with a different name.
        Hide
        Eric Yang added a comment -

        +1 looks good.

        Show
        Eric Yang added a comment - +1 looks good.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496341/HADOOP-7681-1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/231//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/231//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496341/HADOOP-7681-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/231//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/231//console This message is automatically generated.
        Hide
        Matt Foley added a comment -

        Committed to 0.20-sec and 0.20.205. Thanks Arpit! And thanks Eric for the review.

        Show
        Matt Foley added a comment - Committed to 0.20-sec and 0.20.205. Thanks Arpit! And thanks Eric for the review.
        Hide
        Eric Yang added a comment -

        Arpit, the patch has staled for trunk. Could you generate the patch again without:

        log4j.category.SecurityLogger=${security.audit.logger}
        

        Thanks

        Show
        Eric Yang added a comment - Arpit, the patch has staled for trunk. Could you generate the patch again without: log4j.category.SecurityLogger=${security.audit.logger} Thanks
        Hide
        Arpit Gupta added a comment -

        generated patch for trunk again.

        Show
        Arpit Gupta added a comment - generated patch for trunk again.
        Hide
        Eric Yang added a comment -

        I just committed this to trunk and branch 0.23, thanks Arpit.

        Show
        Eric Yang added a comment - I just committed this to trunk and branch 0.23, thanks Arpit.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1011 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1011/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1011 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1011/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1089 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1089/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1089 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1089/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1030 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1030/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1030 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1030/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #821 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/821/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #821 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/821/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/30/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178862
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/30/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178862 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/37/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178862
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/37/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178862 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #851 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/851/)
        HADOOP-7681. Fixed security and hdfs audit log4j properties
        (Arpit Gupta via Eric Yang)

        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #851 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/851/ ) HADOOP-7681 . Fixed security and hdfs audit log4j properties (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178863 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Matt Foley added a comment -

        Closed upon release of 0.20.205.0

        Show
        Matt Foley added a comment - Closed upon release of 0.20.205.0

          People

          • Assignee:
            Arpit Gupta
            Reporter:
            Arpit Gupta
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development