Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5040

Audit log for admin commands/ logging output of all DFS admin commands

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.1.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      enable audit log for all the admin commands/also provide ability to log all the admin commands in separate log file, at this point all the logging is displayed on the console.

      1. HDFS-5040.patch
        12 kB
        Shinichi Yamashita
      2. HDFS-5040.patch
        12 kB
        Shinichi Yamashita
      3. HDFS-5040.patch
        13 kB
        Shinichi Yamashita
      4. HDFS-5040.009.patch
        54 kB
        Kuhu Shukla
      5. HDFS-5040.008.patch
        51 kB
        Kuhu Shukla
      6. HDFS-5040.007.patch
        51 kB
        Kuhu Shukla
      7. HDFS-5040.006.patch
        51 kB
        Kuhu Shukla
      8. HDFS-5040.005.patch
        51 kB
        Kuhu Shukla
      9. HDFS-5040.004.patch
        44 kB
        Kuhu Shukla
      10. HDFS-5040.001.patch
        10 kB
        Kuhu Shukla

        Activity

        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Committed to trunk.Raghu C Doppalapudi thanks for reporting and Kuhu Shukla thanks a lot for your contribution,are you plan to raise follow up jira's considering Vinay comment also..

        Vinayakumar B thanks a lot for additional review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Committed to trunk . Raghu C Doppalapudi thanks for reporting and Kuhu Shukla thanks a lot for your contribution,are you plan to raise follow up jira's considering Vinay comment also.. Vinayakumar B thanks a lot for additional review.
        Hide
        brahmareddy Brahma Reddy Battula added a comment - - edited

        TestFailures are unrelated. +1 on latest patch.Going to commit shortly.

        Show
        brahmareddy Brahma Reddy Battula added a comment - - edited TestFailures are unrelated. +1 on latest patch.Going to commit shortly.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 13m 34s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 findbugs 1m 45s trunk passed
        +1 javadoc 0m 42s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 48s the patch passed
        +1 javac 0m 48s the patch passed
        -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 274 unchanged - 10 fixed = 275 total (was 284)
        +1 mvnsite 0m 55s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 47s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 119m 6s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        145m 28s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.TestDFSClientRetries
          hadoop.hdfs.web.TestWebHDFS
          hadoop.hdfs.TestReadStripedFileWithMissingBlocks



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12888566/HDFS-5040.009.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 802d3e7af4be 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4002bf0
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21310/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21310/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21310/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21310/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 34s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 55s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 42s trunk passed       Patch Compile Tests +1 mvninstall 0m 52s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 274 unchanged - 10 fixed = 275 total (was 284) +1 mvnsite 0m 55s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 119m 6s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 145m 28s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.TestReadStripedFileWithMissingBlocks Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12888566/HDFS-5040.009.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 802d3e7af4be 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4002bf0 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21310/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21310/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21310/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21310/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Added audit logging to getDatanodeStorageReport() and added a test as well. Just one operationName for all kinds of reports. Let me know if that looks ok. Thanks a lot for the comments/review.

        Show
        kshukla Kuhu Shukla added a comment - Added audit logging to getDatanodeStorageReport() and added a test as well. Just one operationName for all kinds of reports. Let me know if that looks ok. Thanks a lot for the comments/review.
        Hide
        vinayrpet Vinayakumar B added a comment -

        I noticed some of the existing admin commands (most are related to ec) logs audit as 'fail' for non-accessControlExceptions in finally block.
        in HDFS-9395 it was decided to log failed audit logs for only on AccessControlExceptions, and success audit logs on successfull operation. Not in finally block.
        Anyway, this is not from the current patch. Can be handled in separate Jira.

        1. getDatanodeStorageReport() is not logging the audit. Though its not available as a command, stil user can access DFSClient's API to invoke it. I think its better to have audit for this as well. Currently balancer will use this API.

        +1 once this is done.

        Show
        vinayrpet Vinayakumar B added a comment - I noticed some of the existing admin commands (most are related to ec) logs audit as 'fail' for non-accessControlExceptions in finally block. in HDFS-9395 it was decided to log failed audit logs for only on AccessControlExceptions, and success audit logs on successfull operation. Not in finally block. Anyway, this is not from the current patch. Can be handled in separate Jira. 1. getDatanodeStorageReport() is not logging the audit. Though its not available as a command, stil user can access DFSClient's API to invoke it. I think its better to have audit for this as well. Currently balancer will use this API. +1 once this is done.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        +1,will commit on next week as I want somebody else also look into this. Test failures are unrelated as you noted.

        Hope you will raise follow up jira for earlier comments.

        Show
        brahmareddy Brahma Reddy Battula added a comment - +1,will commit on next week as I want somebody else also look into this. Test failures are unrelated as you noted. Hope you will raise follow up jira for earlier comments.
        Hide
        kshukla Kuhu Shukla added a comment -

        Test failures are related to bind exceptions and tests like testZeroByteBlockRecovery etc. are failing with or without the patch.

        Show
        kshukla Kuhu Shukla added a comment - Test failures are related to bind exceptions and tests like testZeroByteBlockRecovery etc. are failing with or without the patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 13m 38s trunk passed
        +1 compile 0m 50s trunk passed
        +1 checkstyle 0m 39s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 findbugs 1m 38s trunk passed
        +1 javadoc 0m 41s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 56s the patch passed
        +1 compile 0m 51s the patch passed
        +1 javac 0m 51s the patch passed
        +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 274 unchanged - 8 fixed = 274 total (was 282)
        +1 mvnsite 0m 52s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 105m 21s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        131m 17s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180
          hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110
          hadoop.hdfs.server.namenode.TestReencryptionWithKMS
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
          hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090
          hadoop.hdfs.TestLeaseRecoveryStriped
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.server.namenode.ha.TestHAAppend
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
        Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886651/HDFS-5040.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e189f8772393 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e74d1be
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21094/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21094/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21094/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 38s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 0m 53s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 41s trunk passed       Patch Compile Tests +1 mvninstall 0m 56s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 274 unchanged - 8 fixed = 274 total (was 282) +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 105m 21s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 131m 17s Reason Tests Failed junit tests hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110   hadoop.hdfs.server.namenode.TestReencryptionWithKMS   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886651/HDFS-5040.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e189f8772393 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e74d1be Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21094/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21094/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21094/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Apologies for the delay Brahma Reddy Battula, I somehow missed the last ping. Attaching revised patch with test changes.

        Show
        kshukla Kuhu Shukla added a comment - Apologies for the delay Brahma Reddy Battula , I somehow missed the last ping. Attaching revised patch with test changes.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Would any one else like to chime in? Kuhu Shukla do update the patch..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Would any one else like to chime in? Kuhu Shukla do update the patch..?
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Kuhu Shukla Nice and very neat work here. Appreciate it!. Latest patch almost good to me. apart following minor nits from test.

        1) Instead of following in TestAuditLogger#testWebHdfsAuditLogger, can you reset the log count like testAuditLoggerWithSetPermission..?

        170	      assertEquals("getfileinfo", DummyAuditLogger.lastCommand);
        171	      int logCount = DummyAuditLogger.logCount;
        

        Just add this

             cluster.waitClusterUp();
              DummyAuditLogger.resetLogCount();
        

        2) TestNameNodeMXBean can we handle like this..?

        if (opType.equals(TopConf.ALL_CMDS)) {
                    expected = 2 * NUM_OPS + 2;
                  } else if (opType.equals("datanodeReport")) {
                    expected = 2;
                  } else {
        

        Would any one else like to chime in? Thanks

        Show
        brahmareddy Brahma Reddy Battula added a comment - Kuhu Shukla Nice and very neat work here. Appreciate it!. Latest patch almost good to me. apart following minor nits from test. 1) Instead of following in TestAuditLogger#testWebHdfsAuditLogger , can you reset the log count like testAuditLoggerWithSetPermission ..? 170 assertEquals( "getfileinfo" , DummyAuditLogger.lastCommand); 171 int logCount = DummyAuditLogger.logCount; Just add this cluster.waitClusterUp(); DummyAuditLogger.resetLogCount(); 2) TestNameNodeMXBean can we handle like this..? if (opType.equals(TopConf.ALL_CMDS)) { expected = 2 * NUM_OPS + 2; } else if (opType.equals( "datanodeReport" )) { expected = 2; } else { Would any one else like to chime in? Thanks
        Hide
        kshukla Kuhu Shukla added a comment -

        Brahma Reddy Battula, thank you for the reviews. Request for one more round.

        Show
        kshukla Kuhu Shukla added a comment - Brahma Reddy Battula , thank you for the reviews. Request for one more round.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 16m 12s trunk passed
        +1 compile 0m 59s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 5s trunk passed
        -1 findbugs 2m 1s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 45s trunk passed
              Patch Compile Tests
        +1 mvninstall 1m 1s the patch passed
        +1 compile 0m 58s the patch passed
        +1 javac 0m 58s the patch passed
        +1 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 277 unchanged - 8 fixed = 277 total (was 285)
        +1 mvnsite 1m 3s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 2s the patch passed
        +1 javadoc 0m 43s the patch passed
              Other Tests
        -1 unit 74m 18s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        104m 37s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881285/HDFS-5040.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 3ff42984b63c 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 312e57b
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20638/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20638/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20638/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20638/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 16m 12s trunk passed +1 compile 0m 59s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 5s trunk passed -1 findbugs 2m 1s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 45s trunk passed       Patch Compile Tests +1 mvninstall 1m 1s the patch passed +1 compile 0m 58s the patch passed +1 javac 0m 58s the patch passed +1 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 277 unchanged - 8 fixed = 277 total (was 285) +1 mvnsite 1m 3s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 2s the patch passed +1 javadoc 0m 43s the patch passed       Other Tests -1 unit 74m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 104m 37s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881285/HDFS-5040.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3ff42984b63c 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312e57b Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20638/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/20638/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20638/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20638/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Fixing checkstyle nits. Test failures are not related. Same goes for findbugs.

        Show
        kshukla Kuhu Shukla added a comment - Fixing checkstyle nits. Test failures are not related. Same goes for findbugs.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 14m 53s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 0m 54s trunk passed
        -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 40s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 49s the patch passed
        +1 javac 0m 49s the patch passed
        -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 277 unchanged - 8 fixed = 281 total (was 285)
        +1 mvnsite 0m 54s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 49s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 69m 23s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        96m 38s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881054/HDFS-5040.006.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux dda31a644add 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ec69414
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20622/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20622/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 14m 53s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 0m 54s trunk passed -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 52s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 277 unchanged - 8 fixed = 281 total (was 285) +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 69m 23s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 96m 38s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881054/HDFS-5040.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dda31a644add 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ec69414 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20622/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20622/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Fixing minor checkstyle warnings. Findbug warnings are unrelated and so are the test failures.

        Show
        kshukla Kuhu Shukla added a comment - Fixing minor checkstyle warnings. Findbug warnings are unrelated and so are the test failures.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 13m 52s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 38s trunk passed
        +1 mvnsite 0m 55s trunk passed
        -1 findbugs 1m 39s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
        +1 javadoc 0m 40s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 46s the patch passed
        +1 javac 0m 46s the patch passed
        -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 64 new + 277 unchanged - 8 fixed = 341 total (was 285)
        +1 mvnsite 0m 51s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 36s the patch passed
              Other Tests
        -1 unit 65m 3s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        90m 48s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881040/HDFS-5040.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5cc61baa07d6 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 63cfcb9
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20618/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20618/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 52s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 0m 55s trunk passed -1 findbugs 1m 39s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 49s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 64 new + 277 unchanged - 8 fixed = 341 total (was 285) +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 36s the patch passed       Other Tests -1 unit 65m 3s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 90m 48s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881040/HDFS-5040.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5cc61baa07d6 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 63cfcb9 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20618/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20618/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20618/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Add fail assertion for exception cases. you can use ContractTestUtils.fail

        If I understood that correctly, I have added try-catch for allowed=true cases to fail if any exception is encountered. I have used the Assert.fail() call as the previous patch.
        Fixed the test failures in TestAuditLogger, TestDFSAdminWithHA and TestNameNodeMXBean.
        Other failures are unrelated and flaky.
        Will wait on precommit before asking for further review. Thanks Brahma Reddy Battula for the reviews so far.

        Show
        kshukla Kuhu Shukla added a comment - Add fail assertion for exception cases. you can use ContractTestUtils.fail If I understood that correctly, I have added try-catch for allowed=true cases to fail if any exception is encountered. I have used the Assert.fail() call as the previous patch. Fixed the test failures in TestAuditLogger, TestDFSAdminWithHA and TestNameNodeMXBean. Other failures are unrelated and flaky. Will wait on precommit before asking for further review. Thanks Brahma Reddy Battula for the reviews so far.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 16m 25s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 53s trunk passed
        -1 findbugs 1m 40s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 0m 42s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 46s the patch passed
        +1 javac 0m 46s the patch passed
        -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 62 new + 249 unchanged - 8 fixed = 311 total (was 257)
        +1 mvnsite 0m 52s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 38s the patch passed
              Other Tests
        -1 unit 67m 27s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        95m 53s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestAuditLogger
          hadoop.hdfs.tools.TestDFSAdminWithHA
          hadoop.hdfs.server.namenode.TestNameNodeMXBean
          hadoop.hdfs.server.namenode.TestStorageRestore
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
          hadoop.hdfs.server.namenode.TestCheckpoint



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879873/HDFS-5040.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 85f31b90726e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f9139ac
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20533/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20533/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 16m 25s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 53s trunk passed -1 findbugs 1m 40s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 42s trunk passed       Patch Compile Tests +1 mvninstall 0m 49s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 62 new + 249 unchanged - 8 fixed = 311 total (was 257) +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 67m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 95m 53s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150   hadoop.hdfs.server.namenode.TestCheckpoint Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12879873/HDFS-5040.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 85f31b90726e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f9139ac Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20533/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20533/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20533/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Kuhu Shukla thanks for updating the patch.

        1.looks you missed to give operationName for rollEditLog

        4519	      writeUnlock();
        

        2. Add fail assertion for exception cases. you can use ContractTestUtils.fail
        3. Use GenericTestUtils.assertExceptionContains for asserting the messages
        4. looks all the tests are failing (may some problem with my env..?),will wait for jenkin results.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Kuhu Shukla thanks for updating the patch. 1.looks you missed to give operationName for rollEditLog 4519 writeUnlock(); 2. Add fail assertion for exception cases. you can use ContractTestUtils.fail 3. Use GenericTestUtils.assertExceptionContains for asserting the messages 4. looks all the tests are failing (may some problem with my env..?),will wait for jenkin results.
        Hide
        kshukla Kuhu Shukla added a comment -

        Added tests and incorporated the review changes. Attaching revised patch.

        Show
        kshukla Kuhu Shukla added a comment - Added tests and incorporated the review changes. Attaching revised patch.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        AUDITLOG.info(AUTH_SUCCESSFUL_FOR + user);

        This is connection level , not for every operation.

        I am thinking, something like below.

        Change logAuditEvent(...) to default in FSNamesystem.java

        -  private void logAuditEvent(boolean succeeded, String cmd, String src)
        +  void logAuditEvent(boolean succeeded, String cmd, String src)
        

        call in NameNodeRpcServer.java as follows.

        +    namesystem.logAuditEvent(true,"refreshSuperUserGroupsConfiguration",null);
        
        Show
        brahmareddy Brahma Reddy Battula added a comment - AUDITLOG.info(AUTH_SUCCESSFUL_FOR + user); This is connection level , not for every operation. I am thinking, something like below. Change logAuditEvent(...) to default in FSNamesystem.java - private void logAuditEvent( boolean succeeded, String cmd, String src) + void logAuditEvent( boolean succeeded, String cmd, String src) call in NameNodeRpcServer.java as follows. + namesystem.logAuditEvent( true , "refreshSuperUserGroupsConfiguration" , null );
        Hide
        kshukla Kuhu Shukla added a comment -

        Thank you Brahma Reddy Battula.

        can we cover success cases here..?

        I think we cover this in Server code as follows, in addition to failure cases as well,(the format is not like audit log though).

         AUDITLOG.info(AUTH_SUCCESSFUL_FOR + user);
        
        Show
        kshukla Kuhu Shukla added a comment - Thank you Brahma Reddy Battula . can we cover success cases here..? I think we cover this in Server code as follows, in addition to failure cases as well,(the format is not like audit log though). AUDITLOG.info(AUTH_SUCCESSFUL_FOR + user);
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        I think therefore we don't need to cover these as part of this change.

        can we cover success cases here..?

        Should this be done separately since commands like listCorruptFileBlocks is not an admin command?

        ok

        Show
        brahmareddy Brahma Reddy Battula added a comment - I think therefore we don't need to cover these as part of this change. can we cover success cases here..? Should this be done separately since commands like listCorruptFileBlocks is not an admin command? ok
        Hide
        kshukla Kuhu Shukla added a comment -

        Also Check for usages of checkSuperuserPrivilege(), wherever this is called, add audit logs for that RPC if not already there.also move checkSuperuserPrivilege() check even before obtaining any lock.

        Should this be done separately since commands like listCorruptFileBlocks is not an admin command?

        Show
        kshukla Kuhu Shukla added a comment - Also Check for usages of checkSuperuserPrivilege(), wherever this is called, add audit logs for that RPC if not already there.also move checkSuperuserPrivilege() check even before obtaining any lock. Should this be done separately since commands like listCorruptFileBlocks is not an admin command?
        Hide
        kshukla Kuhu Shukla added a comment -

        For refreshServiceAcl, refreshUserToGroupsMappings, refreshSuperUserGroupsConfiguration, refreshCallQueue, refresh, we handle super user checks at the Protocol level by

        @KerberosInfo(
        serverPrincipal=CommonConfigurationKeys.HADOOP_SECURITY_SERVICE_USER_NAME_KEY)
        

        I think therefore we don't need to cover these as part of this change. Appreciate any comments on this, Brahma Reddy Battula.

        Show
        kshukla Kuhu Shukla added a comment - For refreshServiceAcl , refreshUserToGroupsMappings , refreshSuperUserGroupsConfiguration , refreshCallQueue , refresh , we handle super user checks at the Protocol level by @KerberosInfo( serverPrincipal=CommonConfigurationKeys.HADOOP_SECURITY_SERVICE_USER_NAME_KEY) I think therefore we don't need to cover these as part of this change. Appreciate any comments on this, Brahma Reddy Battula .
        Hide
        kshukla Kuhu Shukla added a comment -

        Thank you so much Brahma Reddy Battula for the detailed review/comments.

        I feel those also can be handled as part of this jira and DN commands we might need discuss further and handle in seperate jira..?

        Agreed. Will post an updated patch asap.

        Hope you'll add testcases.

        Yes, I am working on it right now and wanted to get some comments on the approach through this patch.
        Will update patch soon.

        Show
        kshukla Kuhu Shukla added a comment - Thank you so much Brahma Reddy Battula for the detailed review/comments. I feel those also can be handled as part of this jira and DN commands we might need discuss further and handle in seperate jira..? Agreed. Will post an updated patch asap. Hope you'll add testcases. Yes, I am working on it right now and wanted to get some comments on the approach through this patch. Will update patch soon.
        Hide
        brahmareddy Brahma Reddy Battula added a comment - - edited

        Comments for patch:

        1. ) Instead of checkPermissionsAndLogAuditEvent, can we name it as checkSuperuserPrivilege(String operationName) since this check will be for super user..?
        2. ) looks you missed success audit log for finalizeUpgrade,only updated the failure case(ACE).
        3. ) a)
          4342	    String operationName = "safemode";

          can we have something like below,to know the exact action..?
          String opName = action.toString().toLowerCase();
          b) we might need to handle default: case also..? so better introduce one boolean variable..?

        4. ) checkSuperuserPrivilege() just move out of the writelock.
          6162	      try {
          6163	        checkOperation(OperationCategory.WRITE);
          6164	        checkNameNodeSafeMode("Cannot allow snapshot for " + path);
          6165	        checkSuperuserPrivilege();
          6166	        FSDirSnapshotOp.allowSnapshot(dir, snapshotManager, path);
          6167	        success = true;
          6168	      } finally {
          6169	        writeUnlock(operationName);
          6170	      }
          6171	    } catch (AccessControlException ace) {
          6172	      logAuditEvent(success, operationName, path, null, null);
          6173	      throw ace;
          
        5. ) Quota commands needs to handle for specific command.Like below.
          private String getQuotaCommand(long nsQuota, long dsQuota) {
              if (nsQuota == HdfsConstants.QUOTA_RESET
                  && dsQuota == HdfsConstants.QUOTA_DONT_SET) {
                return "clearQuota";
              } else if (nsQuota == HdfsConstants.QUOTA_DONT_SET
                  && dsQuota == HdfsConstants.QUOTA_RESET) {
                return "clearSpaceQuota";
              } else if (dsQuota == HdfsConstants.QUOTA_DONT_SET) {
                return "setQuota";
              } else {
                return "setSpaceQuota";
              }
            }
        6. ) Also Check for usages of checkSuperuserPrivilege(), wherever this is called, add audit logs for that RPC if not already there.also move checkSuperuserPrivilege() check even before obtaining any lock.
        7. ) Hope you'll add testcases.
        Show
        brahmareddy Brahma Reddy Battula added a comment - - edited Comments for patch: ) Instead of checkPermissionsAndLogAuditEvent , can we name it as checkSuperuserPrivilege(String operationName) since this check will be for super user..? ) looks you missed success audit log for finalizeUpgrade ,only updated the failure case(ACE). ) a) 4342 String operationName = "safemode" ; can we have something like below,to know the exact action..? String opName = action.toString().toLowerCase(); b) we might need to handle default: case also..? so better introduce one boolean variable..? ) checkSuperuserPrivilege() just move out of the writelock. 6162 try { 6163 checkOperation(OperationCategory.WRITE); 6164 checkNameNodeSafeMode( "Cannot allow snapshot for " + path); 6165 checkSuperuserPrivilege(); 6166 FSDirSnapshotOp.allowSnapshot(dir, snapshotManager, path); 6167 success = true ; 6168 } finally { 6169 writeUnlock(operationName); 6170 } 6171 } catch (AccessControlException ace) { 6172 logAuditEvent(success, operationName, path, null , null ); 6173 throw ace; ) Quota commands needs to handle for specific command.Like below. private String getQuotaCommand( long nsQuota, long dsQuota) { if (nsQuota == HdfsConstants.QUOTA_RESET && dsQuota == HdfsConstants.QUOTA_DONT_SET) { return "clearQuota" ; } else if (nsQuota == HdfsConstants.QUOTA_DONT_SET && dsQuota == HdfsConstants.QUOTA_RESET) { return "clearSpaceQuota" ; } else if (dsQuota == HdfsConstants.QUOTA_DONT_SET) { return "setQuota" ; } else { return "setSpaceQuota" ; } } ) Also Check for usages of checkSuperuserPrivilege() , wherever this is called, add audit logs for that RPC if not already there.also move checkSuperuserPrivilege() check even before obtaining any lock. ) Hope you'll add testcases.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Kuhu Shukla thanks for working on this. As of now, following are admin commands, please correct me if I miss anything.

        The following commands do not tie to a call in FSNameSystem directly as of today and I am looking for some suggestions on if and which ones of these would need to be covered as part of this change.

        Some commands are directly from NameNodeRpcServer(listed below),I feel those also can be handled as part of this jira and DN commands we might need discuss further and handle in seperate jira..?

        command Already Handled Need to Handle Reamarks
        [-report [-live] [-dead] [-decommissioning] [-enteringmaintenance] [-inmaintenance]]
        NO YES FSNamesystem#datanodeReport
        [-safemode <enter | leave | get | wait>]
        NO YES
        [-saveNamespace [-beforeShutdown]]
        NO YES
        [-rollEdits]
        NO YES
        [-restoreFailedStorage true|false|check]
        NO YES
        [-refreshNodes]
        NO YES
        [-setQuota <quota> <dirname>...<dirname>]
        YES YES Need to differentiate for each quota command
        [-clrQuota <dirname>...<dirname>]
        YES YES Need to differentiate for each quota command
        [-setSpaceQuota <quota> [-storageType <storagetype>] <dirname>...<dirname>]
        YES YES Need to differentiate for each quota command
        [-clrSpaceQuota [-storageType <storagetype>] <dirname>...<dirname>]
        YES YES Need to differentiate for each quota command
        [-finalizeUpgrade]
        NO YES
        [-rollingUpgrade [<query|prepare|finalize>]]
        NO YES
        [-refreshServiceAcl]
        NO YES NameNodeRpcServer#refreshServiceAcl
        [-refreshUserToGroupsMappings]
        NO YES NameNodeRpcServer#refreshUserToGroupsMappings
        [-refreshSuperUserGroupsConfiguration]
        NO YES NameNodeRpcServer#refreshSuperUserGroupsConfiguration
        [-refreshCallQueue]
        NO YES NameNodeRpcServer#refreshCallQueue
        [-refresh <host:ipc_port> <key> [arg1..argn]
        NO YES NameNodeRpcServer#refresh
        [-reconfig <namenode|datanode> <host:ipc_port> <start|status|properties>]
        NO YES namenode call part can be NameNodeRpcServer#startReconfiguration,getReconfigurationStatus,listReconfigurableProperties
        [-printTopology]
        NO YES Covered as part of report
        [-refreshNamenodes datanode_host:ipc_port]
        NO DN Cmd
        [-getVolumeReport datanode_host:ipc_port]
        NO DN Cmd
        [-deleteBlockPool datanode_host:ipc_port blockpoolId [force]]
        NO DN Cmd
        [-setBalancerBandwidth <bandwidth in bytes per second>]
        NO YES
        [-getBalancerBandwidth <datanode_host:ipc_port>]
        NO DN Cmd
        [-fetchImage <local directory>]
        NO NA
        [-allowSnapshot <snapshotDir>]
        YES NO
        [-disallowSnapshot <snapshotDir>]
        YES NO
        [-shutdownDatanode <datanode_host:ipc_port> [upgrade]]
        NO DN Cmd
        [-evictWriters <datanode_host:ipc_port>]
        NO DN
        [-getDatanodeInfo <datanode_host:ipc_port>]
        NO DN Cmd
        [-metasave filename]
        NO YES
        [-triggerBlockReport [-incremental] <datanode_host:ipc_port>]
        NO DN Cmd
        [-listOpenFiles]
        YES NO
        [-help [cmd]]
        NO NA
        Show
        brahmareddy Brahma Reddy Battula added a comment - Kuhu Shukla thanks for working on this. As of now, following are admin commands, please correct me if I miss anything. The following commands do not tie to a call in FSNameSystem directly as of today and I am looking for some suggestions on if and which ones of these would need to be covered as part of this change. Some commands are directly from NameNodeRpcServer(listed below),I feel those also can be handled as part of this jira and DN commands we might need discuss further and handle in seperate jira..? command Already Handled Need to Handle Reamarks [-report [-live] [-dead] [-decommissioning] [-enteringmaintenance] [-inmaintenance]] NO YES FSNamesystem#datanodeReport [-safemode <enter | leave | get | wait>] NO YES [-saveNamespace [-beforeShutdown]] NO YES [-rollEdits] NO YES [-restoreFailedStorage true|false|check] NO YES [-refreshNodes] NO YES [-setQuota <quota> <dirname>...<dirname>] YES YES Need to differentiate for each quota command [-clrQuota <dirname>...<dirname>] YES YES Need to differentiate for each quota command [-setSpaceQuota <quota> [-storageType <storagetype>] <dirname>...<dirname>] YES YES Need to differentiate for each quota command [-clrSpaceQuota [-storageType <storagetype>] <dirname>...<dirname>] YES YES Need to differentiate for each quota command [-finalizeUpgrade] NO YES [-rollingUpgrade [<query|prepare|finalize>]] NO YES [-refreshServiceAcl] NO YES NameNodeRpcServer#refreshServiceAcl [-refreshUserToGroupsMappings] NO YES NameNodeRpcServer#refreshUserToGroupsMappings [-refreshSuperUserGroupsConfiguration] NO YES NameNodeRpcServer#refreshSuperUserGroupsConfiguration [-refreshCallQueue] NO YES NameNodeRpcServer#refreshCallQueue [-refresh <host:ipc_port> <key> [arg1..argn] NO YES NameNodeRpcServer#refresh [-reconfig <namenode|datanode> <host:ipc_port> <start|status|properties>] NO YES namenode call part can be NameNodeRpcServer#startReconfiguration,getReconfigurationStatus,listReconfigurableProperties [-printTopology] NO YES Covered as part of report [-refreshNamenodes datanode_host:ipc_port] NO DN Cmd [-getVolumeReport datanode_host:ipc_port] NO DN Cmd [-deleteBlockPool datanode_host:ipc_port blockpoolId [force]] NO DN Cmd [-setBalancerBandwidth <bandwidth in bytes per second>] NO YES [-getBalancerBandwidth <datanode_host:ipc_port>] NO DN Cmd [-fetchImage <local directory>] NO NA [-allowSnapshot <snapshotDir>] YES NO [-disallowSnapshot <snapshotDir>] YES NO [-shutdownDatanode <datanode_host:ipc_port> [upgrade]] NO DN Cmd [-evictWriters <datanode_host:ipc_port>] NO DN [-getDatanodeInfo <datanode_host:ipc_port>] NO DN Cmd [-metasave filename] NO YES [-triggerBlockReport [-incremental] <datanode_host:ipc_port>] NO DN Cmd [-listOpenFiles] YES NO [-help [cmd]] NO NA
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
              trunk Compile Tests
        +1 mvninstall 13m 40s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 38s trunk passed
        +1 mvnsite 0m 53s trunk passed
        -1 findbugs 1m 49s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 0m 42s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 36s the patch passed
        +1 mvnsite 0m 54s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 58s the patch passed
        +1 javadoc 0m 41s the patch passed
              Other Tests
        -1 unit 94m 28s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        120m 44s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010
          hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
          hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
          hadoop.hdfs.TestRollingUpgrade
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-5040
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12877848/HDFS-5040.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e317d76406a4 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0b7afc0
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20328/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/20328/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20328/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20328/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests +1 mvninstall 13m 40s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 0m 53s trunk passed -1 findbugs 1m 49s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 42s trunk passed       Patch Compile Tests +1 mvninstall 0m 52s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 41s the patch passed       Other Tests -1 unit 94m 28s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 120m 44s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.server.namenode.TestReconstructStripedBlocks   hadoop.hdfs.TestRollingUpgrade Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-5040 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12877848/HDFS-5040.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e317d76406a4 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0b7afc0 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/20328/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/20328/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20328/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20328/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        First cut of the patch that audit logs allowed=false for AcessControlException and true for successful cases for a subset of the admin commands. The following commands do not tie to a call in FSNameSystem directly as of today and I am looking for some suggestions on if and which ones of these would need to be covered as part of this change. Appreciate any comments on this.

        -getDatanodeInfo
        -triggerBlockReport
        -help
        -fetchImage
        -deleteBlockPool
        -printTopology
        reconfig
        -refreshCallQueue
        -refreshSuperUserGroupsConfiguration
        -refreshUserToGroupsMappings
        -refreshServiceAcl
        -report
        
        Show
        kshukla Kuhu Shukla added a comment - First cut of the patch that audit logs allowed=false for AcessControlException and true for successful cases for a subset of the admin commands. The following commands do not tie to a call in FSNameSystem directly as of today and I am looking for some suggestions on if and which ones of these would need to be covered as part of this change. Appreciate any comments on this. -getDatanodeInfo -triggerBlockReport -help -fetchImage -deleteBlockPool -printTopology reconfig -refreshCallQueue -refreshSuperUserGroupsConfiguration -refreshUserToGroupsMappings -refreshServiceAcl -report
        Hide
        kshukla Kuhu Shukla added a comment -

        Brahma Reddy Battula, sure. Will update patch in a couple days. Let me know if that would be ok. Thanks a lot!

        Show
        kshukla Kuhu Shukla added a comment - Brahma Reddy Battula , sure. Will update patch in a couple days. Let me know if that would be ok. Thanks a lot!
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Kuhu Shukla thanks for working on this.

        It's good to have audit log for all the admin commands. can you update the patch...?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Kuhu Shukla thanks for working on this. It's good to have audit log for all the admin commands. can you update the patch...?
        Hide
        sinchii Shinichi Yamashita added a comment -

        Kuhu Shukla Thank you for your message. I hand over this ticket to you.

        Show
        sinchii Shinichi Yamashita added a comment - Kuhu Shukla Thank you for your message. I hand over this ticket to you.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 4s HDFS-5040 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch
        JIRA Issue HDFS-5040
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14561/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s HDFS-5040 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch JIRA Issue HDFS-5040 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14561/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Shinichi Yamashita, If you are not currently working on this, may I assign this to myself and continue the work as a follow up of HDFS-9395. Thanks a lot!

        Show
        kshukla Kuhu Shukla added a comment - Shinichi Yamashita , If you are not currently working on this, may I assign this to myself and continue the work as a follow up of HDFS-9395 . Thanks a lot!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10571/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10571/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5144//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5144//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607420/HDFS-5040.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5144//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5144//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12607320/HDFS-5040.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestAuditLogger

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607320/HDFS-5040.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestAuditLogger +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5136//console This message is automatically generated.
        Hide
        sinchii Shinichi Yamashita added a comment -

        I attach a patch file which I revised.

        Show
        sinchii Shinichi Yamashita added a comment - I attach a patch file which I revised.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12607273/HDFS-5040.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5130//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607273/HDFS-5040.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5130//console This message is automatically generated.
        Hide
        sinchii Shinichi Yamashita added a comment -

        I attach a patch file related to previous comment.

        Show
        sinchii Shinichi Yamashita added a comment - I attach a patch file related to previous comment.
        Hide
        sinchii Shinichi Yamashita added a comment -

        It seems that "allowSnapshot" and "disallowSnapshot" have been already implemented in trunk.
        And it is possible to let NameNode's audit log output the remaining commands except "refreshNameNodes" and "deleteBlockPool".

        Show
        sinchii Shinichi Yamashita added a comment - It seems that "allowSnapshot" and "disallowSnapshot" have been already implemented in trunk. And it is possible to let NameNode's audit log output the remaining commands except "refreshNameNodes" and "deleteBlockPool".

          People

          • Assignee:
            kshukla Kuhu Shukla
            Reporter:
            raghu.hbase@gmail.com Raghu C Doppalapudi
          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development