Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8316

Audit logging should be disabled by default

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 2.0.0-alpha
        • Fix Version/s: 2.0.2-alpha
        • Component/s: conf
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          HADOOP-7633 made hdfs, mr and security audit logging on by default (INFO level) in log4j.properties used for the packages, this then got copied over to the non-packaging log4j.properties in HADOOP-8216 (which made them consistent).

          Seems like we should keep with the v1.x setting which is disabled (WARNING level) by default. There's a performance overhead to audit logging, and HADOOP-7633 provided not rationale (just "We should add the audit logs as part of default confs") as to why they were enabled for the packages.

            Issue Links

            is related to

            Bug - A problem which impairs or prevents the functions of the product. HADOOP-7633 log4j.properties should be added to the hadoop conf on deploy

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              eli2 Eli Collins added a comment -

              Patch attached.

              • update hadoop.security.logger to NullAppender in log4j.properties, it is set explicitly by default in the bin and env scripts, so this is mostly a nop
              • hdfs/mapred.audit.logger and now default to the NullAppender in log4j.properties. Update hdfs.audit.logger in hadoop-env.sh to match. This is being made configurable in HADOOP-8224. mapred.audit.logger is not set in the bin or env scripts and is dead code, filed HADOOP-8392 for that (and to hookup RM/NM).

              Testing, verified the hdfs audit log is no longer automatically created and logged to when run from a tarball install.

              Show
              eli2 Eli Collins added a comment - Patch attached. update hadoop.security.logger to NullAppender in log4j.properties, it is set explicitly by default in the bin and env scripts, so this is mostly a nop hdfs/mapred.audit.logger and now default to the NullAppender in log4j.properties. Update hdfs.audit.logger in hadoop-env.sh to match. This is being made configurable in HADOOP-8224 . mapred.audit.logger is not set in the bin or env scripts and is dead code, filed HADOOP-8392 for that (and to hookup RM/NM). Testing, verified the hdfs audit log is no longer automatically created and logged to when run from a tarball install.
              Hide
              Permalink
              hadoopqa Hadoop QA added a comment -

              -1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12526538/hadoop-8316.txt
              against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              -1 tests included. The patch doesn't appear to include any new or modified tests.
              Please justify why no new tests are needed for this patch.
              Also please list what manual steps were performed to verify this patch.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 javadoc. The javadoc tool did not generate any warning messages.

              +1 eclipse:eclipse. The patch built with eclipse:eclipse.

              +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

              +1 contrib tests. The patch passed contrib unit tests.

              Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/984//testReport/
              Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/984//console

              This message is automatically generated.

              Show
              hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526538/hadoop-8316.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/984//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/984//console This message is automatically generated.
              Hide
              Permalink
              phunt Patrick Hunt added a comment -

              Reviewed the patch, looks good to me.

              Show
              phunt Patrick Hunt added a comment - Reviewed the patch, looks good to me.
              Hide
              Permalink
              phunt Patrick Hunt added a comment -

              +1

              Show
              phunt Patrick Hunt added a comment - +1
              Hide
              Permalink
              eli2 Eli Collins added a comment -

              I've committed this and merged to branch-2.

              Show
              eli2 Eli Collins added a comment - I've committed this and merged to branch-2.
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-Hdfs-trunk-Commit #2307 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2307/)
              HADOOP-8316. Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334)

              Result = SUCCESS
              eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334
              Files :

              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Show
              hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2307 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2307/ ) HADOOP-8316 . Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-Common-trunk-Commit #2233 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2233/)
              HADOOP-8316. Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334)

              Result = SUCCESS
              eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334
              Files :

              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Show
              hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2233 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2233/ ) HADOOP-8316 . Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk-Commit #2250 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2250/)
              HADOOP-8316. Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334)

              Result = ABORTED
              eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334
              Files :

              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Show
              hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2250 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2250/ ) HADOOP-8316 . Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334) Result = ABORTED eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-Hdfs-trunk #1041 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1041/)
              HADOOP-8316. Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334)

              Result = FAILURE
              eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334
              Files :

              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Show
              hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1041 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1041/ ) HADOOP-8316 . Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk #1077 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1077/)
              HADOOP-8316. Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334)

              Result = SUCCESS
              eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334
              Files :

              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh
              • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
              Show
              hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1077 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1077/ ) HADOOP-8316 . Audit logging should be disabled by default. Contributed by Eli Collins (Revision 1337334) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337334 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties

                People

                • Assignee:
                  eli Eli Collins
                  Reporter:
                  eli2 Eli Collins
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development