Details

    • Hadoop Flags:
      Reviewed

      Description

      This proposes adding a bunch of unit tests for command 'dfsadmin setSpaceQuota' and 'dfsadmin clrSpaceQuota'.
      1. test to set space quote using negative number.
      2. test to set and clear space quote, regular usage.
      3. test to set and clear space quote by storage type.
      4. test to set and clear space quote when directory doesn't exist.
      5. test to set and clear space quote when path is a file.
      6. test to set and clear space quote when user has no access right.

      1. HDFS-11011.000.patch
        14 kB
        Xiaobing Zhou
      2. HDFS-11011.001.patch
        13 kB
        Xiaobing Zhou
      3. HDFS-11011.002.patch
        14 kB
        Xiaobing Zhou
      4. HDFS-11011.003.patch
        14 kB
        Xiaobing Zhou
      5. HDFS-11011.004.patch
        13 kB
        Mingliang Liu
      6. HDFS-11011.005.patch
        13 kB
        Xiaobing Zhou
      7. HDFS-11011.006.patch
        13 kB
        Xiaobing Zhou
      8. HDFS-11011.007.patch
        13 kB
        Xiaobing Zhou
      9. HDFS-11011.008.patch
        89 kB
        Xiaobing Zhou

        Issue Links

          Activity

          Hide
          xiaobingo Xiaobing Zhou added a comment -

          v000 patch is posted, please kindly review it, thanks.

          Show
          xiaobingo Xiaobing Zhou added a comment - v000 patch is posted, please kindly review it, thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 49s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 49s trunk passed
          +1 javadoc 0m 42s trunk passed
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 1 unchanged - 0 fixed = 7 total (was 1)
          +1 mvnsite 0m 53s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 1s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 60m 38s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          79m 50s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833302/HDFS-11011.000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 122cd60caa5f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d9f73f1
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17150/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17150/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17150/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17150/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 49s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 1 unchanged - 0 fixed = 7 total (was 1) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 1s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 60m 38s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 79m 50s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833302/HDFS-11011.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 122cd60caa5f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d9f73f1 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17150/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17150/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17150/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17150/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          v001 fixed check style issues.

          Show
          xiaobingo Xiaobing Zhou added a comment - v001 fixed check style issues.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 44s trunk passed
          +1 compile 0m 54s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 1m 2s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 57s trunk passed
          +1 javadoc 0m 41s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 49s the patch passed
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 58s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 64m 55s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          86m 41s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestErasureCodeBenchmarkThroughput
            hadoop.hdfs.server.datanode.TestDataNodeLifeline
            hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833434/HDFS-11011.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8b82255d6e62 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0007360
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17166/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17166/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17166/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 44s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 57s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 64m 55s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 86m 41s Reason Tests Failed junit tests hadoop.hdfs.TestErasureCodeBenchmarkThroughput   hadoop.hdfs.server.datanode.TestDataNodeLifeline   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833434/HDFS-11011.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8b82255d6e62 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0007360 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17166/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17166/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17166/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Hi Xiaobing, before reviewing the patch, I have a question: testQuotaCommands#testQuotaCommands is testing the commands; are you adding new tests, or they are duplicate? This seems also true to HDFS-11016.

          Show
          liuml07 Mingliang Liu added a comment - Hi Xiaobing, before reviewing the patch, I have a question: testQuotaCommands#testQuotaCommands is testing the commands; are you adding new tests, or they are duplicate? This seems also true to HDFS-11016 .
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          Hi Mingliang, they are kind of overlapped, but here it also tested unauthorized access and verified CLI outputs.

          Show
          xiaobingo Xiaobing Zhou added a comment - Hi Mingliang, they are kind of overlapped, but here it also tested unauthorized access and verified CLI outputs.
          Hide
          liuml07 Mingliang Liu added a comment -

          Can we move them together (and add comments in the UT about related places)?

          Show
          liuml07 Mingliang Liu added a comment - Can we move them together (and add comments in the UT about related places)?
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          v002 moved tests to TestQuote.

          Show
          xiaobingo Xiaobing Zhou added a comment - v002 moved tests to TestQuote.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks for your patch Xiaobing Zhou. It looks good to me overall.

          TestDFSAdmin.java
          68	/**
          69	 * set/clrSpaceQuote are tested in {@link org.apache.hadoop.hdfs.TestQuota}.
          70	 */
          

          Add one more javadoc for the test class itself? Or this line will be treated as the javadoc for the test class.

          /**
           * Test cases for <code>hdfs dfsadmin<code> command.
           *
           * Tests for some subcommands  are covered elewhere, e.g. set/clrSpaceQuote are tested in {@link org.apache.hadoop.hdfs.TestQuota}.
           */
          
          	1482	    /* set space quota */
          1483	    resetStream();
          1484	    outs.clear();
          1485	    ret = whoever.doAs(new PrivilegedExceptionAction<Integer>() {
          1486	      @Override
          1487	      public Integer run() throws Exception {
          1488	        return ToolRunner.run(
          1489	            admin,
          1490	            new String[] {"-setSpaceQuota", "2048", dir.toString()});
          1491	      }
          1492	    });
          1493	    assertEquals(-1, ret);
          1494	    scanIntoList(err, outs);
          1495	    assertThat(outs.get(0),
          1496	        is(allOf(containsString("setSpaceQuota"),
          1497	            containsString("Access denied for user whoever"),
          1498	            containsString("Superuser privilege is required"))));
          1499	
          1500	    /* clear space quota */
          1501	    resetStream();
          1502	    outs.clear();
          1503	    ret = whoever.doAs(new PrivilegedExceptionAction<Integer>() {
          1504	      @Override
          1505	      public Integer run() throws Exception {
          1506	        return ToolRunner.run(
          1507	            admin,
          1508	            new String[] {"-clrSpaceQuota", dir.toString()});
          1509	      }
          1510	    });
          1511	    assertEquals(-1, ret);
          1512	    scanIntoList(err, outs);
          1513	    assertThat(outs.get(0),
          1514	        is(allOf(containsString("clrSpaceQuota"),
          1515	            containsString("Access denied for user whoever"),
          1516	            containsString("Superuser privilege is required"))));
          1517	  }
          

          Code duplicates. Can you use a for loop? See https://github.com/apache/hadoop/blob/129125404244f35ee63b8f0491a095371685e9ba/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSAdmin.java#L212-L212 This applies to other places too.

          By the way, can you update the descriptions with all the newly add test cases? (Simply copy the javadoc for each test case and list them here will work just fine.)

          Show
          liuml07 Mingliang Liu added a comment - Thanks for your patch Xiaobing Zhou . It looks good to me overall. TestDFSAdmin.java 68 /** 69 * set/clrSpaceQuote are tested in {@link org.apache.hadoop.hdfs.TestQuota}. 70 */ Add one more javadoc for the test class itself? Or this line will be treated as the javadoc for the test class. /** * Test cases for <code>hdfs dfsadmin<code> command. * * Tests for some subcommands are covered elewhere, e.g. set/clrSpaceQuote are tested in {@link org.apache.hadoop.hdfs.TestQuota}. */ 1482 /* set space quota */ 1483 resetStream(); 1484 outs.clear(); 1485 ret = whoever.doAs( new PrivilegedExceptionAction< Integer >() { 1486 @Override 1487 public Integer run() throws Exception { 1488 return ToolRunner.run( 1489 admin, 1490 new String [] { "-setSpaceQuota" , "2048" , dir.toString()}); 1491 } 1492 }); 1493 assertEquals(-1, ret); 1494 scanIntoList(err, outs); 1495 assertThat(outs.get(0), 1496 is(allOf(containsString( "setSpaceQuota" ), 1497 containsString( "Access denied for user whoever" ), 1498 containsString( "Superuser privilege is required" )))); 1499 1500 /* clear space quota */ 1501 resetStream(); 1502 outs.clear(); 1503 ret = whoever.doAs( new PrivilegedExceptionAction< Integer >() { 1504 @Override 1505 public Integer run() throws Exception { 1506 return ToolRunner.run( 1507 admin, 1508 new String [] { "-clrSpaceQuota" , dir.toString()}); 1509 } 1510 }); 1511 assertEquals(-1, ret); 1512 scanIntoList(err, outs); 1513 assertThat(outs.get(0), 1514 is(allOf(containsString( "clrSpaceQuota" ), 1515 containsString( "Access denied for user whoever" ), 1516 containsString( "Superuser privilege is required" )))); 1517 } Code duplicates. Can you use a for loop? See https://github.com/apache/hadoop/blob/129125404244f35ee63b8f0491a095371685e9ba/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSAdmin.java#L212-L212 This applies to other places too. By the way, can you update the descriptions with all the newly add test cases? (Simply copy the javadoc for each test case and list them here will work just fine.)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 11s trunk passed
          +1 compile 0m 46s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 38s trunk passed
          +1 mvninstall 0m 47s the patch passed
          +1 compile 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 16 new + 38 unchanged - 1 fixed = 54 total (was 39)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 57m 15s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          76m 17s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestFSImageWithAcl
            hadoop.hdfs.TestDFSShell



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833831/HDFS-11011.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux adb8294ca65d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f5d9235
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17191/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17191/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17191/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17191/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 11s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 16 new + 38 unchanged - 1 fixed = 54 total (was 39) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 57m 15s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 76m 17s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.TestDFSShell Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833831/HDFS-11011.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux adb8294ca65d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f5d9235 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17191/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17191/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17191/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17191/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaobingo Xiaobing Zhou added a comment - - edited

          v003 patch is posted, thanks Mingliang Liu. Due to differences of various commands, it's hard to use a unanimous for-loop.

          1. refactored code for reusability
          2. added Javadoc
          3. fixed check style issues.

          Show
          xiaobingo Xiaobing Zhou added a comment - - edited v003 patch is posted, thanks Mingliang Liu . Due to differences of various commands, it's hard to use a unanimous for-loop. 1. refactored code for reusability 2. added Javadoc 3. fixed check style issues.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 33s trunk passed
          +1 compile 0m 45s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 49s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 38s trunk passed
          +1 mvninstall 0m 44s the patch passed
          +1 compile 0m 41s the patch passed
          +1 javac 0m 41s the patch passed
          -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 38 unchanged - 1 fixed = 46 total (was 39)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 37s the patch passed
          +1 unit 56m 58s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          74m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834033/HDFS-11011.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ca51ef92f664 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6c348c5
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17209/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17209/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17209/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 33s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 49s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 44s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 38 unchanged - 1 fixed = 46 total (was 39) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 56m 58s hadoop-hdfs in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 74m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834033/HDFS-11011.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ca51ef92f664 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6c348c5 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17209/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17209/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17209/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v3 patch looks good to me overall.

          1454	        new String[] {
          1455	            "It should be one line error message like:"
          1456	                + " clrSpaceQuota...Access denied for user...",
          1457	            "clrSpaceQuota",
          1458	            "Access denied for user"});
          

          For var args, we don't need to construct the String[] every time. Just pass as many parameters as we wish.

          runAndVerifyQuota()
          ...
          1539	    /* verify outputs */
          1540	    scanIntoList(ioBuf, outs);
          1541	    assertEquals(msgs[0], expectedOutNum, outs.size());
          1542	    if (expectedOutNum > 0) {
          1543	      assertThat(outs.get(0),
          1544	          is(allOf(containsString(msgs[1]),
          1545	              containsString(msgs[2]))));
          1546	    }
          1547	  }
          

          This indicates that if expectedOutNum is greater than zero, the msgs will always have at least three parameters. This is error-prone to use. Meanwhile, the first string aka msg[0] is the assertEquals failing message, different from other msgs. Let's split them.

          I updated the patch slightly and upload a v4 version. Can you review that, Xiaobing Zhou? Thanks,

          Show
          liuml07 Mingliang Liu added a comment - The v3 patch looks good to me overall. 1454 new String [] { 1455 "It should be one line error message like:" 1456 + " clrSpaceQuota...Access denied for user..." , 1457 "clrSpaceQuota" , 1458 "Access denied for user" }); For var args, we don't need to construct the String[] every time. Just pass as many parameters as we wish. runAndVerifyQuota() ... 1539 /* verify outputs */ 1540 scanIntoList(ioBuf, outs); 1541 assertEquals(msgs[0], expectedOutNum, outs.size()); 1542 if (expectedOutNum > 0) { 1543 assertThat(outs.get(0), 1544 is(allOf(containsString(msgs[1]), 1545 containsString(msgs[2])))); 1546 } 1547 } This indicates that if expectedOutNum is greater than zero, the msgs will always have at least three parameters. This is error-prone to use. Meanwhile, the first string aka msg[0] is the assertEquals failing message, different from other msgs. Let's split them. I updated the patch slightly and upload a v4 version. Can you review that, Xiaobing Zhou ? Thanks,
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 56s trunk passed
          +1 compile 0m 47s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 40s trunk passed
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 40s the patch passed
          +1 javac 0m 40s the patch passed
          -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 38 unchanged - 1 fixed = 47 total (was 39)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 36s the patch passed
          -1 unit 59m 53s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          78m 23s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834269/HDFS-11011.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 36f17a36a154 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e9c4616
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17226/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17226/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17226/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17226/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 56s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 38 unchanged - 1 fixed = 47 total (was 39) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 36s the patch passed -1 unit 59m 53s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 78m 23s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834269/HDFS-11011.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 36f17a36a154 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e9c4616 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17226/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17226/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17226/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17226/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          Thank you Mingliang Liu for the patch. I posted another patch v005 which is based on v002. The reason is that a jumbo reusable function (e.g. v003 and v004) makes code less readable and maintainable.

          Show
          xiaobingo Xiaobing Zhou added a comment - Thank you Mingliang Liu for the patch. I posted another patch v005 which is based on v002. The reason is that a jumbo reusable function (e.g. v003 and v004) makes code less readable and maintainable.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 19s trunk passed
          +1 compile 0m 53s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 51s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 55s the patch passed
          +1 compile 0m 51s the patch passed
          +1 javac 0m 51s the patch passed
          -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 19 new + 38 unchanged - 1 fixed = 57 total (was 39)
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 58s the patch passed
          +1 javadoc 0m 40s the patch passed
          -1 unit 67m 0s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          88m 10s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834575/HDFS-11011.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f28c7974d34c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 262827c
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17243/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17243/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17243/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17243/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 19s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 55s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 19 new + 38 unchanged - 1 fixed = 57 total (was 39) +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 40s the patch passed -1 unit 67m 0s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 88m 10s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834575/HDFS-11011.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f28c7974d34c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 262827c Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17243/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17243/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17243/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17243/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          +1 after the checkstyle warning is addressed. Thanks.

          Show
          liuml07 Mingliang Liu added a comment - +1 after the checkstyle warning is addressed. Thanks.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          Posted v006 patch to fix check style issues. Thanks for reviews.

          Show
          xiaobingo Xiaobing Zhou added a comment - Posted v006 patch to fix check style issues. Thanks for reviews.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 48s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 23s trunk passed
          +1 compile 0m 55s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 0m 42s trunk passed
          +1 mvninstall 0m 47s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 38 unchanged - 1 fixed = 44 total (was 39)
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 50s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 62m 23s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          83m 42s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestMaintenanceState



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834604/HDFS-11011.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4bfc910f8a4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 754cb4e
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17248/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17248/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17248/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 48s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 23s trunk passed +1 compile 0m 55s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 38 unchanged - 1 fixed = 44 total (was 39) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 62m 23s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 83m 42s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834604/HDFS-11011.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4bfc910f8a4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 754cb4e Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17248/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17248/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17248/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          There were still some check styles missed, v007 fixed them.

          Show
          xiaobingo Xiaobing Zhou added a comment - There were still some check styles missed, v007 fixed them.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Xiaobing Zhou and Mingliang Liu for working on this. The v007 looks good to me overall. I just have two questions:

          1. TestQuota#setUp() is introduced with @Before to start a new cluster instance for each test and assigned to TestQuota#dfsCluster. However, there are a few existing test still using separate local DFSMinicluster instance such as TestQuota#testSpaceCommands, TestQuota#testQuotaByStorageType, etc., can you fix them to use TestQuota#dfsCluster to avoid unnecessary mini cluster creation?

          2. Glad to see new tests are added with dedicated test paths. Can you try to run the suite with a single MiniCluster instance that is created with @BeforeClass to save spinup/shutdown cluster for each tests?

                final Path parent = new Path(
                  PathUtils.getTestPath(getClass()),
                  GenericTestUtils.getMethodName()); 
          
          Show
          xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou and Mingliang Liu for working on this. The v007 looks good to me overall. I just have two questions: 1. TestQuota#setUp() is introduced with @Before to start a new cluster instance for each test and assigned to TestQuota#dfsCluster. However, there are a few existing test still using separate local DFSMinicluster instance such as TestQuota#testSpaceCommands, TestQuota#testQuotaByStorageType, etc., can you fix them to use TestQuota#dfsCluster to avoid unnecessary mini cluster creation? 2. Glad to see new tests are added with dedicated test paths. Can you try to run the suite with a single MiniCluster instance that is created with @BeforeClass to save spinup/shutdown cluster for each tests? final Path parent = new Path( PathUtils.getTestPath(getClass()), GenericTestUtils.getMethodName());
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 56s trunk passed
          +1 compile 0m 47s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 0m 42s trunk passed
          +1 mvninstall 0m 51s the patch passed
          +1 compile 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          +1 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39)
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 51s the patch passed
          +1 javadoc 0m 39s the patch passed
          -1 unit 79m 12s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          100m 36s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestPersistBlocks



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11011
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834704/HDFS-11011.007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 48dcc03d04a1 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f63cd78
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17253/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17253/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17253/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 56s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed +1 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39) +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 79m 12s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 100m 36s Reason Tests Failed junit tests hadoop.hdfs.TestPersistBlocks Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11011 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834704/HDFS-11011.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 48dcc03d04a1 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f63cd78 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17253/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17253/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17253/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaobingo Xiaobing Zhou added a comment - - edited

          Thank you Xiaoyu Yao for reviews. v008 removed all mini clusters from test functions except

          testMultipleFilesSmallerThanOneBlock

          which needs specific conf setting. Instead of launching mini cluster per test, a shared cluster accelerates tests significantly.

          Show
          xiaobingo Xiaobing Zhou added a comment - - edited Thank you Xiaoyu Yao for reviews. v008 removed all mini clusters from test functions except testMultipleFilesSmallerThanOneBlock which needs specific conf setting. Instead of launching mini cluster per test, a shared cluster accelerates tests significantly.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Xiaobing Zhou for the update. Patch v008 looks pretty good to me. +1. I will commit it shortly.

          The unit test failure below is unrelated. Seems like a DNS issue, can you open a new ticket for it as my brief search did not find an open ticket for it?

          org.apache.hadoop.hdfs.TestPersistBlocks.TestRestartDfsWithFlush
          End of File Exception between local host is: "48dcc03d04a1/172.17.0.3"; destination host is: "localhost":39481; : java.io.EOFException; For more details see:  http://wiki.apache.org/hadoop/EOFException
          
          
          Show
          xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou for the update. Patch v008 looks pretty good to me. +1. I will commit it shortly. The unit test failure below is unrelated. Seems like a DNS issue, can you open a new ticket for it as my brief search did not find an open ticket for it? org.apache.hadoop.hdfs.TestPersistBlocks.TestRestartDfsWithFlush End of File Exception between local host is: "48dcc03d04a1/172.17.0.3" ; destination host is: "localhost" :39481; : java.io.EOFException; For more details see: http: //wiki.apache.org/hadoop/EOFException
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Xiaobing Zhou for the contribution and Mingliang Liu for the reviews. I've commit the latest patch to trunk, branch-2 and branch-2.8.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou for the contribution and Mingliang Liu for the reviews. I've commit the latest patch to trunk, branch-2 and branch-2.8.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          Xiaoyu Yao, I filed HDFS-11052 to track the flaky failure. Thanks.

          Show
          xiaobingo Xiaobing Zhou added a comment - Xiaoyu Yao , I filed HDFS-11052 to track the flaky failure. Thanks.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10675 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10675/)
          HDFS-11011. Add unit tests for HDFS command 'dfsadmin (xyao: rev 9a8a3864584cec4b545fc974baa110c6839946d4)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSAdmin.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10675 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10675/ ) HDFS-11011 . Add unit tests for HDFS command 'dfsadmin (xyao: rev 9a8a3864584cec4b545fc974baa110c6839946d4) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSAdmin.java

            People

            • Assignee:
              xiaobingo Xiaobing Zhou
              Reporter:
              xiaobingo Xiaobing Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development