Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Specific HDFS ops can be selectively excluded from audit logging via 'dfs.namenode.audit.log.debug.cmdlist' configuration.

      Description

      The audit logging of getFileInfo() was added in HDFS-3733. Since this is a one of the most called method, users have noticed that audit log is now filled with this. Since we now have HTTP request logging, this seems unnecessary.

      1. HDFS-6888.008.patch
        9 kB
        Chen He
      2. HDFS-6888.07.patch
        9 kB
        Arpit Agarwal
      3. HDFS-6888.patch
        0.7 kB
        Chen He
      4. HDFS-6888-2.patch
        7 kB
        Chen He
      5. HDFS-6888-3.patch
        10 kB
        Chen He
      6. HDFS-6888-4.patch
        10 kB
        Chen He
      7. HDFS-6888-5.patch
        10 kB
        Chen He
      8. HDFS-6888-6.patch
        10 kB
        Chen He

        Issue Links

          Activity

          Hide
          kihwal Kihwal Lee added a comment -

          See the patch in HDFS-3733. When this logging was added, a number of test cases were added. These test cases need to be modified. "open" (i.e. getBlockLocations()) is logged for webhdfs. For example, {{testAuditWebHdfsOpen()} generates 3 audit log entries: getfileinfo/webhdfs, open/webhdfs and open/rpc. After this change, it will have the last two entries.

          Show
          kihwal Kihwal Lee added a comment - See the patch in HDFS-3733 . When this logging was added, a number of test cases were added. These test cases need to be modified. "open" (i.e. getBlockLocations()) is logged for webhdfs. For example, {{testAuditWebHdfsOpen()} generates 3 audit log entries: getfileinfo/webhdfs, open/webhdfs and open/rpc. After this change, it will have the last two entries.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          getfileinfo a common RPC but we have identified many times using the audit log an app that needs some "help" in optimizing their FileSystem API usage.

          Show
          jira.shegalov Gera Shegalov added a comment - getfileinfo a common RPC but we have identified many times using the audit log an app that needs some "help" in optimizing their FileSystem API usage.
          Hide
          kihwal Kihwal Lee added a comment -

          What if we log it at debug level?

          Show
          kihwal Kihwal Lee added a comment - What if we log it at debug level?
          Hide
          kihwal Kihwal Lee added a comment -

          We could have logAuditEvent() check cmd against "getfileinfo" or a collection of such commands and log at debug level. Otherwise we will have to change HdfsAuditLogger, which will introduce incompatibility. Users who want to see this in the log can change the log level at runtime.

          Show
          kihwal Kihwal Lee added a comment - We could have logAuditEvent() check cmd against "getfileinfo" or a collection of such commands and log at debug level. Otherwise we will have to change HdfsAuditLogger, which will introduce incompatibility. Users who want to see this in the log can change the log level at runtime.
          Hide
          airbots Chen He added a comment -

          patch updated. Thank you for your comments. Kihwal Lee and Gera Shegalov.

          Show
          airbots Chen He added a comment - patch updated. Thank you for your comments. Kihwal Lee and Gera Shegalov .
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12663508/HDFS-6888-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 3 release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
          org.apache.hadoop.hdfs.server.namenode.TestAuditLogger
          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          org.apache.hadoop.hdfs.TestDecommission

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12663508/HDFS-6888-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 3 release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract org.apache.hadoop.hdfs.server.namenode.TestAuditLogger org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover org.apache.hadoop.hdfs.TestDecommission +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7703//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12663508/HDFS-6888-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 3 release audit warnings.

          -1 core tests. The test build failed in hadoop-hdfs-project/hadoop-hdfs

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12663508/HDFS-6888-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 3 release audit warnings. -1 core tests . The test build failed in hadoop-hdfs-project/hadoop-hdfs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7710//console This message is automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Kihwal Lee, I am +1 for making some commands debug level, so we have an option to capture them in the logs and remove them.
          Chen He, how about making the list of DEBUG-level commands configurable via a csv list for conf.getTrimmedStrings instead of hardcoding it as in v2 of the patch.

          Show
          jira.shegalov Gera Shegalov added a comment - Kihwal Lee , I am +1 for making some commands debug level, so we have an option to capture them in the logs and remove them. Chen He , how about making the list of DEBUG-level commands configurable via a csv list for conf.getTrimmedStrings instead of hardcoding it as in v2 of the patch.
          Hide
          airbots Chen He added a comment -

          Hi Gera Shegalov, you lost me there. We need to let the getfileinfo cmd only log when the auditlog is in debug level, right?

          Show
          airbots Chen He added a comment - Hi Gera Shegalov , you lost me there. We need to let the getfileinfo cmd only log when the auditlog is in debug level, right?
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Chen He, sorry for being unclear.

          Kihwal Lee suggests:

          We could have logAuditEvent() check cmd against "getfileinfo" or a collection of such commands and log at debug level.

          Picking this idea up can you introduce some conf like dfs.audit.<loglevel>.cmdlist=getfileinfo,anotherLogFloodingCmd,...

          In o.a.h.hdfs.server.namenode.FSNamesystem.DefaultAuditLogger#initialize you could read the list using auditDebugCmds conf.getTrimmedStrings("dfs.audit.debug.cmdlist" and use it for filtering. Currenly v2 hardcodes "getfileinfo".

          --- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          +++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          @@ -359,6 +359,9 @@ private void logAuditEvent(boolean succeeded,
                 UserGroupInformation ugi, InetAddress addr, String cmd, String src,
                 String dst, HdfsFileStatus stat) {
               FileStatus status = null;
          +    if(cmd.equals("getfileinfo") && !auditLog.isDebugEnabled()) {
          +      return;
          +    }
               if (stat != null) {
                 Path symlink = stat.isSymlink() ? new Path(stat.getSymlink()) : null;
                 Path path = dst != null ? new Path(dst) : new Path(src);
          

          Also auditLog.isDebugEnabled() is a cheaper check, and should be done before equals

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Chen He , sorry for being unclear. Kihwal Lee suggests: We could have logAuditEvent() check cmd against "getfileinfo" or a collection of such commands and log at debug level. Picking this idea up can you introduce some conf like dfs.audit.<loglevel>.cmdlist=getfileinfo,anotherLogFloodingCmd,... In o.a.h.hdfs.server.namenode.FSNamesystem.DefaultAuditLogger#initialize you could read the list using auditDebugCmds conf.getTrimmedStrings("dfs.audit.debug.cmdlist" and use it for filtering. Currenly v2 hardcodes "getfileinfo". --- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java +++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java @@ -359,6 +359,9 @@ private void logAuditEvent( boolean succeeded, UserGroupInformation ugi, InetAddress addr, String cmd, String src, String dst, HdfsFileStatus stat) { FileStatus status = null ; + if (cmd.equals( "getfileinfo" ) && !auditLog.isDebugEnabled()) { + return ; + } if (stat != null ) { Path symlink = stat.isSymlink() ? new Path(stat.getSymlink()) : null ; Path path = dst != null ? new Path(dst) : new Path(src); Also auditLog.isDebugEnabled() is a cheaper check, and should be done before equals
          Hide
          airbots Chen He added a comment -

          patch updated.

          Show
          airbots Chen He added a comment - patch updated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org
          against trunk revision c4c9a78.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7820//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org against trunk revision c4c9a78. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7820//console This message is automatically generated.
          Hide
          airbots Chen He added a comment -

          It works on my mac. Is this because Hadoop repository move to git?

          Show
          airbots Chen He added a comment - It works on my mac. Is this because Hadoop repository move to git?
          Hide
          airbots Chen He added a comment -

          HEAD is now at c4c9a78 HADOOP-11001. Fix test-patch to work with the git repo. (kasha)
          Removing patchprocess/
          Switched to branch 'trunk'
          Your branch is up-to-date with 'origin/trunk'.
          Current branch trunk is up to date.
          /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory
          /bin/grep: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory
          /bin/grep: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory
          HDFS-6888 patch is being downloaded at Thu Aug 28 14:11:35 UTC 2014 from
          http://issues.apache.org
          /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory
          cp: cannot stat '/home/jenkins/buildSupport/lib/*': No such file or directory
          /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 44: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory
          /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 100: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory
          /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 102: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory
          The patch does not appear to apply with p0 to p2
          rm: cannot remove '/tmp/tmp.paths.7370': No such file or directory
          PATCH APPLICATION FAILED

          Show
          airbots Chen He added a comment - HEAD is now at c4c9a78 HADOOP-11001 . Fix test-patch to work with the git repo. (kasha) Removing patchprocess/ Switched to branch 'trunk' Your branch is up-to-date with 'origin/trunk'. Current branch trunk is up to date. /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory /bin/grep: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory /bin/grep: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/jira: No such file or directory HDFS-6888 patch is being downloaded at Thu Aug 28 14:11:35 UTC 2014 from http://issues.apache.org /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory cp: cannot stat '/home/jenkins/buildSupport/lib/*': No such file or directory /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 44: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 100: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/dev-support/smart-apply-patch.sh: line 102: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/patch: No such file or directory The patch does not appear to apply with p0 to p2 rm: cannot remove '/tmp/tmp.paths.7370': No such file or directory PATCH APPLICATION FAILED
          Hide
          airbots Chen He added a comment -

          retrigger QA

          Show
          airbots Chen He added a comment - retrigger QA
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12664998/HDFS-6888-3.patch
          against trunk revision c4c9a78.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.security.TestRefreshUserMappings

          The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestDecommission

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7829//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7829//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12664998/HDFS-6888-3.patch against trunk revision c4c9a78. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.security.TestRefreshUserMappings The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestDecommission +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7829//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7829//console This message is automatically generated.
          Hide
          airbots Chen He added a comment -

          Test error is because of HADOOP-11020

          Show
          airbots Chen He added a comment - Test error is because of HADOOP-11020
          Hide
          airbots Chen He added a comment -

          retrigger QA

          Show
          airbots Chen He added a comment - retrigger QA
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12667859/HDFS-6888-3.patch
          against trunk revision b02a4b4.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7988//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7988//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12667859/HDFS-6888-3.patch against trunk revision b02a4b4. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7988//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7988//console This message is automatically generated.
          Hide
          airbots Chen He added a comment -

          unit test failure is because of HDFS-6694

          Show
          airbots Chen He added a comment - unit test failure is because of HDFS-6694
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Thanks, Chen He for updating the patch. +1 (non-binding)

          Two nits:
          debugCmdSet can be declared as a general Set

          private Set<String> debugCmdSet = new HashSet<String>();
          

          Its initialization can be more brief:

            debugCmdSet.addAll(Arrays.asList(conf.getTrimmedStrings(
                DFSConfigKeys.DFS_AUDIT_LOG_DEBUG_CMDLIST)));
          
          Show
          jira.shegalov Gera Shegalov added a comment - Thanks, Chen He for updating the patch. +1 (non-binding) Two nits: debugCmdSet can be declared as a general Set private Set< String > debugCmdSet = new HashSet< String >(); Its initialization can be more brief: debugCmdSet.addAll(Arrays.asList(conf.getTrimmedStrings( DFSConfigKeys.DFS_AUDIT_LOG_DEBUG_CMDLIST)));
          Hide
          airbots Chen He added a comment -

          patch updated.

          Show
          airbots Chen He added a comment - patch updated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12670807/HDFS-6888-4.patch
          against trunk revision f48686a.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8171//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8171//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12670807/HDFS-6888-4.patch against trunk revision f48686a. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8171//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8171//console This message is automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          Chen He, thank you for fixing this issue. +1 (non-binding) on the patch.

          I have one very tiny nit and one question:

          • In the description of hdfs-default.xml, I would have used the phrase "writes to" instead of "outputs"
          • Is there someplace that documents all of the configuration properties? Is that generated automatically from the descriptions in the *xml files?
          Show
          eepayne Eric Payne added a comment - Chen He , thank you for fixing this issue. +1 (non-binding) on the patch. I have one very tiny nit and one question: In the description of hdfs-default.xml , I would have used the phrase "writes to" instead of "outputs" Is there someplace that documents all of the configuration properties? Is that generated automatically from the descriptions in the *xml files?
          Hide
          airbots Chen He added a comment -

          Thank you for the review, Eric Payne. I change "outputs" to "writes to" in the latest patch.
          For the documentation changes, I am not sure. But, I did some research in the source code:

          There are 3 hdfs-default.xml files:
          ./hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          ./hadoop-hdfs-project/hadoop-hdfs/src/site/resources/hdfs-default.xml
          ./hadoop-hdfs-project/hadoop-hdfs/target/classes/hdfs-default.xml

          The 3rd one is from the 1st one (same contents). But it looks like the 2nd one is for webpage (site package). However, the 2nd one has more 500 lines difference between 1st one. To confirm which one is used by current hadoop.com, I checked http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/core-default.xml, The contents are more closer to the 1st one (or maybe 3rd one).

          Then, I guess that we may not need to modify the 2nd one which is "./hadoop-hdfs-project/hadoop-hdfs/src/site/resources/hdfs-default.xml".

          Show
          airbots Chen He added a comment - Thank you for the review, Eric Payne . I change "outputs" to "writes to" in the latest patch. For the documentation changes, I am not sure. But, I did some research in the source code: There are 3 hdfs-default.xml files: ./hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml ./hadoop-hdfs-project/hadoop-hdfs/src/site/resources/hdfs-default.xml ./hadoop-hdfs-project/hadoop-hdfs/target/classes/hdfs-default.xml The 3rd one is from the 1st one (same contents). But it looks like the 2nd one is for webpage (site package). However, the 2nd one has more 500 lines difference between 1st one. To confirm which one is used by current hadoop.com, I checked http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/core-default.xml , The contents are more closer to the 1st one (or maybe 3rd one). Then, I guess that we may not need to modify the 2nd one which is "./hadoop-hdfs-project/hadoop-hdfs/src/site/resources/hdfs-default.xml".
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12672571/HDFS-6888-5.patch
          against trunk revision c2fa5d1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          org.apache.hadoop.hdfs.server.balancer.TestBalancer
          org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12672571/HDFS-6888-5.patch against trunk revision c2fa5d1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8299//console This message is automatically generated.
          Hide
          airbots Chen He added a comment -

          update patch against trunk.TestBalancer and TestFailureToReadEdits work fine on my machine. TestPipelinesFailover failure is because of HDFS-6694

          Show
          airbots Chen He added a comment - update patch against trunk.TestBalancer and TestFailureToReadEdits work fine on my machine. TestPipelinesFailover failure is because of HDFS-6694
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12676436/HDFS-6888-6.patch
          against trunk revision a36399e.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8483//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8483//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676436/HDFS-6888-6.patch against trunk revision a36399e. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8483//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8483//console This message is automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          One of the key points of the HDFS audit log is to show accesses to files, including for security purposes. If a user can legitimately use getFileInfo(), then it needs to get logged.

          Show
          aw Allen Wittenauer added a comment - One of the key points of the HDFS audit log is to show accesses to files, including for security purposes. If a user can legitimately use getFileInfo(), then it needs to get logged.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 28s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 7m 54s The applied patch generated 2 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 11s Pre-build of native portion
          -1 hdfs tests 163m 59s Tests failed in hadoop-hdfs.
              212m 16s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12676436/HDFS-6888-6.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / a00e001
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10395/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10395/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10395/testReport/
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10395/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 28s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 7m 54s The applied patch generated 2 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 163m 59s Tests failed in hadoop-hdfs.     212m 16s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12676436/HDFS-6888-6.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a00e001 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10395/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10395/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10395/testReport/ Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10395/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          One of the key points of the HDFS audit log is to show accesses to files, including for security purposes. If a user can legitimately use getFileInfo(), then it needs to get logged.

          The patch allows users to exclude certain ops from audit log. getFileIno needs to be removed from hdfs-default.xml, so that it is logged by default. I will update the title.

          Show
          kihwal Kihwal Lee added a comment - One of the key points of the HDFS audit log is to show accesses to files, including for security purposes. If a user can legitimately use getFileInfo(), then it needs to get logged. The patch allows users to exclude certain ops from audit log. getFileIno needs to be removed from hdfs-default.xml , so that it is logged by default. I will update the title.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          This would be nice to have.

          HDFS-6888.07.patch updates Chen He's patch based on Kihwal's comment and adds new test cases. Since the default behavior is unchanged, changes to existing test cases were reverted.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited This would be nice to have. HDFS-6888 .07.patch updates Chen He 's patch based on Kihwal's comment and adds new test cases. Since the default behavior is unchanged, changes to existing test cases were reverted.
          Hide
          airbots Chen He added a comment -

          Thank [~Arpt Agarwal] for the updates. I was planning to do the update tonight. Also, thank Kihwal Lee ,Eric Payne, and Allen Wittenauer reviews.

          Show
          airbots Chen He added a comment - Thank [~Arpt Agarwal] for the updates. I was planning to do the update tonight. Also, thank Kihwal Lee , Eric Payne , and Allen Wittenauer reviews.
          Hide
          vinayrpet Vinayakumar B added a comment -

          This is nice.
          Changes looks nice.

          One nit.
          change the conf key name to dfs.namenode.audit.log.debug.cmdlist to be in sync with other audit confs.

          +1 once addressed.

          Show
          vinayrpet Vinayakumar B added a comment - This is nice. Changes looks nice. One nit. change the conf key name to dfs.namenode.audit.log.debug.cmdlist to be in sync with other audit confs. +1 once addressed.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 7s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 52s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 35s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 16s Pre-build of native portion
          -1 hdfs tests 166m 40s Tests failed in hadoop-hdfs.
              209m 51s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestDecommission
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.tools.TestHdfsConfigFields



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731643/HDFS-6888.07.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0daa5ad
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10974/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10974/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10974/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 52s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 35s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 166m 40s Tests failed in hadoop-hdfs.     209m 51s   Reason Tests Failed unit tests hadoop.hdfs.TestDecommission   hadoop.hdfs.server.mover.TestStorageMover   hadoop.tools.TestHdfsConfigFields Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731643/HDFS-6888.07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0daa5ad hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10974/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10974/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10974/console This message was automatically generated.
          Hide
          airbots Chen He added a comment -

          patch updated. Thank you for the review Vinayakumar B.

          Show
          airbots Chen He added a comment - patch updated. Thank you for the review Vinayakumar B .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 45s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 33s There were no new javac warning messages.
          +1 javadoc 9m 47s There were no new javadoc warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 14s The applied patch generated 2 new checkstyle issues (total was 688, now 689).
          +1 shellcheck 0m 6s There were no new shellcheck (v0.3.3) issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 20s Pre-build of native portion
          -1 hdfs tests 170m 16s Tests failed in hadoop-hdfs.
              213m 44s  



          Reason Tests
          Failed unit tests hadoop.tools.TestHdfsConfigFields



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732884/HDFS-6888.008.patch
          Optional Tests shellcheck javadoc javac unit findbugs checkstyle
          git revision trunk / def9136
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10981/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10981/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10981/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10981/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 45s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 47s There were no new javadoc warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 14s The applied patch generated 2 new checkstyle issues (total was 688, now 689). +1 shellcheck 0m 6s There were no new shellcheck (v0.3.3) issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 20s Pre-build of native portion -1 hdfs tests 170m 16s Tests failed in hadoop-hdfs.     213m 44s   Reason Tests Failed unit tests hadoop.tools.TestHdfsConfigFields Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732884/HDFS-6888.008.patch Optional Tests shellcheck javadoc javac unit findbugs checkstyle git revision trunk / def9136 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10981/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10981/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10981/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10981/console This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Test failure is not related to this jira and its already fixed in HDFS-8371

          +1 for latest patch.
          Will commit soon.

          Show
          vinayrpet Vinayakumar B added a comment - Test failure is not related to this jira and its already fixed in HDFS-8371 +1 for latest patch. Will commit soon.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7839 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7839/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7839 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7839/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          Hide
          vinayrpet Vinayakumar B added a comment -

          Committed to trunk and branch-2.
          Thanks Chen He for the contribution.
          Thanks everyone for reviews and great suggestions.

          Show
          vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2. Thanks Chen He for the contribution. Thanks everyone for reviews and great suggestions.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #928 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/928/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #928 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/928/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #197 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/197/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #197 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/197/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2126 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2126/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2126 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2126/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/186/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/186/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/196/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/196/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2144 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2144/)
          HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2144 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2144/ ) HDFS-6888 . Allow selectively audit logging ops (Contributed by Chen He) (vinayakumarb: rev 3bef7c80a97709b367781180b2e11fc50653d3c8) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLogAtDebug.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java

            People

            • Assignee:
              airbots Chen He
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development