Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8437

getLocalPathForWrite should throw IOException for invalid paths

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 2.6.0, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
      Here it will not thrown any exception but earlier version it used throw.

      1. HADOOP-8437_1.patch
        2 kB
        Brahma Reddy Battula
      2. HADOOP-8437_2.patch
        2 kB
        Brahma Reddy Battula
      3. HADOOP-8437.patch
        2 kB
        Brahma Reddy Battula
      4. HADOOP-8437-003.patch
        2 kB
        Brahma Reddy Battula
      5. HADOOP-8437-004.patch
        2 kB
        Brahma Reddy Battula
      6. HADOOP-8437-005.patch
        2 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          In earlier version

          localDirs = dirs.toArray(new String[dirs.size()]);
          dirDF = dfList.toArray(new DF[dirs.size()]);
          savedLocalDirs = newLocalDirs;
          if (0 == dirs.size()) {
          		throw new IOException("No dirs to select.Total dirs size is 0");
          			}
          // randomize the first disk picked in the round-robin selection
          dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
          dirNumLastAccessedforKnownSize = dirNumLastAccessed;
          

          Here it is throwing IOException saying that No dirs to select.Total dirs size is 0

          But in brnach2 and trunk...code is like follwoing..

          localDirs = dirs.toArray(new String[dirs.size()]);
          dirDF = dfList.toArray(new DF[dirs.size()]);
          savedLocalDirs = newLocalDirs;
            
          // randomize the first disk picked in the round-robin selection 
          dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
          

          Here follwoing check is removed..

          if (0 == dirs.size()) {
          		throw new IOException("No dirs to select.Total dirs size is 0");
          			}
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - In earlier version localDirs = dirs.toArray( new String [dirs.size()]); dirDF = dfList.toArray( new DF[dirs.size()]); savedLocalDirs = newLocalDirs; if (0 == dirs.size()) { throw new IOException( "No dirs to select.Total dirs size is 0" ); } // randomize the first disk picked in the round-robin selection dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size()); dirNumLastAccessedforKnownSize = dirNumLastAccessed; Here it is throwing IOException saying that No dirs to select.Total dirs size is 0 But in brnach2 and trunk...code is like follwoing.. localDirs = dirs.toArray( new String [dirs.size()]); dirDF = dfList.toArray( new DF[dirs.size()]); savedLocalDirs = newLocalDirs; // randomize the first disk picked in the round-robin selection dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size()); Here follwoing check is removed.. if (0 == dirs.size()) { throw new IOException( "No dirs to select.Total dirs size is 0" ); }
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          attaching Patch

          Show
          brahmareddy Brahma Reddy Battula added a comment - attaching Patch
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12529963/HADOOP-8437.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12529963/HADOOP-8437.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//console This message is automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Following testcases failing because someother(timedout) reason not because of this fix..

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:
          
          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - Following testcases failing because someother(timedout) reason not because of this fix.. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
          Hide
          qwertymaniac Harsh J added a comment -

          Hi Brahma,

          Couple of comments:

          +        if (0 == dirs.size()) {
          

          I feel this check can happen slightly more earlier as the hunk itself seems to suggest that dirs is being utilized before we come to this new check.

          +          throw new IOException("No dirs to select.Total dirs size is 0");
          

          Please space the two sentences (general rule: Always a space after a punctuation mark, never before.)

          +    for (int i = 0; i < 256; i++) {
          +      invalidPath.append("A");
          +    }
          

          Seems excessive to do this. Just declare a constant string?

          fail("not throwing the exception");
          

          Lets describe failure strings better since it is a specific test we're carrying out/expecting a failure from: "An exception should have been thrown when an invalid directory is used."

          Thanks!

          Show
          qwertymaniac Harsh J added a comment - Hi Brahma, Couple of comments: + if (0 == dirs.size()) { I feel this check can happen slightly more earlier as the hunk itself seems to suggest that dirs is being utilized before we come to this new check. + throw new IOException( "No dirs to select.Total dirs size is 0" ); Please space the two sentences (general rule: Always a space after a punctuation mark, never before.) + for ( int i = 0; i < 256; i++) { + invalidPath.append( "A" ); + } Seems excessive to do this. Just declare a constant string? fail( "not throwing the exception" ); Lets describe failure strings better since it is a specific test we're carrying out/expecting a failure from: "An exception should have been thrown when an invalid directory is used." Thanks!
          Hide
          qwertymaniac Harsh J added a comment -

          Canceling patch until the few superficial comments are addressed.

          Show
          qwertymaniac Harsh J added a comment - Canceling patch until the few superficial comments are addressed.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Hi Harsh,

          Thanks a lot for reviewing the patch..I updated patch..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Hi Harsh, Thanks a lot for reviewing the patch..I updated patch..
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12546490/HADOOP-8437_1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.TestLocalDirAllocator

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546490/HADOOP-8437_1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestLocalDirAllocator +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//console This message is automatically generated.
          Hide
          qwertymaniac Harsh J added a comment -

          Doesn't seem like the logic is working (and hence the test failure), can you recheck the patch please Brahma? I'm not sure what we're missing

          Show
          qwertymaniac Harsh J added a comment - Doesn't seem like the logic is working (and hence the test failure), can you recheck the patch please Brahma? I'm not sure what we're missing
          Hide
          qwertymaniac Harsh J added a comment -

          Ok so its cause of my earlier comment:

          +    for (int i = 0; i < 256; i++) {
          +      invalidPath.append("A");
          +    }
          

          This creates a condition where the filename request is > max allowed filename length on some FS-es. Since thats not a nice way to do it (it may begin 'passing' on some FSes), lets instead play with permissions or something?

          Show
          qwertymaniac Harsh J added a comment - Ok so its cause of my earlier comment: + for ( int i = 0; i < 256; i++) { + invalidPath.append( "A" ); + } This creates a condition where the filename request is > max allowed filename length on some FS-es. Since thats not a nice way to do it (it may begin 'passing' on some FSes), lets instead play with permissions or something?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Hi Harsh,

          do u have any suggestions on this apart from permissions..? or following will be okie.

           for (int i = 0; i < 256; i++) {
          +      invalidPath.append("A");
          +    }
          

          I tried with special chars which are not supported by OS..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Hi Harsh, do u have any suggestions on this apart from permissions..? or following will be okie. for ( int i = 0; i < 256; i++) { + invalidPath.append( "A" ); + } I tried with special chars which are not supported by OS..
          Hide
          qwertymaniac Harsh J added a comment -

          Would permissions not work?

          I had tried the special char approach on your patch earlier but Linux supports almost everything we could use as a reasonable test. If permission tweaks are not gonna work, I'm fine with the 256 length test, with a comment added along that this depends on the FS in use to fail and may pass on some FS.

          Show
          qwertymaniac Harsh J added a comment - Would permissions not work? I had tried the special char approach on your patch earlier but Linux supports almost everything we could use as a reasonable test. If permission tweaks are not gonna work, I'm fine with the 256 length test, with a comment added along that this depends on the FS in use to fail and may pass on some FS.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Harsh J

          Hope 256 length test is okie..can you check first patch..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Harsh J Hope 256 length test is okie..can you check first patch..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Hi Harsh,

          Attaching patch(As I did not find any other way do the test hence used 256 char test only )

          I think, this we can close...Please have a look once..
          (Sorry for late reply..after long time)..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Hi Harsh, Attaching patch(As I did not find any other way do the test hence used 256 char test only ) I think, this we can close...Please have a look once.. (Sorry for late reply..after long time)..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Harsh J Kindly review the attached patch !!!!

          Show
          brahmareddy Brahma Reddy Battula added a comment - Harsh J Kindly review the attached patch !!!!
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12708073/HADOOP-8437-003.patch
          against trunk revision 232eca9.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708073/HADOOP-8437-003.patch against trunk revision 232eca9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//console This message is automatically generated.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Brahma Reddy Battula thank you for taking this issue. As Harsh J mentioned, we should fix the test case to make it FS independent. Could you try it?

          I'm fine with the 256 length test, with a comment added along that this depends on the FS in use to fail and may pass on some FS.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula thank you for taking this issue. As Harsh J mentioned, we should fix the test case to make it FS independent. Could you try it? I'm fine with the 256 length test, with a comment added along that this depends on the FS in use to fail and may pass on some FS.
          Hide
          zxu zhihai xu added a comment -

          thanks Brahma Reddy Battula for the patch! If the size of dirs is zero, it will cause the following exception.

          java.lang.IllegalArgumentException: n must be positive
          at java.util.Random.nextInt(Random.java:250)
          at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243)
          

          same exception found at HADOOP-6766 and MAPREDUCE-635.
          Can we avoid IllegalArgumentException by checking dirs.size() non-zero before calling dirIndexRandomizer.nextInt with the following simpler change?

          if (dirs.size() > 0) {
            // randomize the first disk picked in the round-robin selection 
            dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
          }
          

          then it will trigger DiskErrorException later due to zero length localDirs if calling from getLocalPathForWrite or getLocalPathToRead .

          Show
          zxu zhihai xu added a comment - thanks Brahma Reddy Battula for the patch! If the size of dirs is zero, it will cause the following exception. java.lang.IllegalArgumentException: n must be positive at java.util.Random.nextInt(Random.java:250) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243) same exception found at HADOOP-6766 and MAPREDUCE-635 . Can we avoid IllegalArgumentException by checking dirs.size() non-zero before calling dirIndexRandomizer.nextInt with the following simpler change? if (dirs.size() > 0) { // randomize the first disk picked in the round-robin selection dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size()); } then it will trigger DiskErrorException later due to zero length localDirs if calling from getLocalPathForWrite or getLocalPathToRead .
          Hide
          zxu zhihai xu added a comment -

          By the way, a much easier way for a test case may be just to change the configuration to a string with only white space.
          For example, run the following code:

              conf1.set(CONTEXT, " ");
              dirAllocator.getLocalPathForWrite("/test", conf1);
          

          You can get the following exception:

          java.lang.IllegalArgumentException: n must be positive
          	at java.util.Random.nextInt(Random.java:300)
          	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:308)
          	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:347)
          	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
          	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
          	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115)
          	at org.apache.hadoop.fs.TestLocalDirAllocator.testShouldNotthrowNPE(TestLocalDirAllocator.java:320)
          
          Show
          zxu zhihai xu added a comment - By the way, a much easier way for a test case may be just to change the configuration to a string with only white space. For example, run the following code: conf1.set(CONTEXT, " " ); dirAllocator.getLocalPathForWrite( "/test" , conf1); You can get the following exception: java.lang.IllegalArgumentException: n must be positive at java.util.Random.nextInt(Random.java:300) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:308) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:347) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.fs.TestLocalDirAllocator.testShouldNotthrowNPE(TestLocalDirAllocator.java:320)
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          zhihai xu thanks for taking a look into this issue.. Updated patch based on your comments..Kindly Review..

          Show
          brahmareddy Brahma Reddy Battula added a comment - zhihai xu thanks for taking a look into this issue.. Updated patch based on your comments..Kindly Review..
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 53s There were no new javac warning messages.
          +1 javadoc 10m 9s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 23, now 24).
          -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 47s Tests passed in hadoop-common.
              48m 31s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764209/HADOOP-8437-004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d6fa34e
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 53s There were no new javac warning messages. +1 javadoc 10m 9s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 23, now 24). -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 47s Tests passed in hadoop-common.     48m 31s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764209/HADOOP-8437-004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d6fa34e Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Uploaded patch for checkstyle and whitespace errors fix..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Uploaded patch for checkstyle and whitespace errors fix..
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 30s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 10m 8s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 10s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 6m 58s Tests failed in hadoop-common.
              48m 1s  



          Reason Tests
          Failed unit tests hadoop.security.ssl.TestReloadingX509TrustManager
          Timed out tests org.apache.hadoop.ha.TestZKFailoverController
            org.apache.hadoop.ha.TestZKFailoverControllerStress
            org.apache.hadoop.metrics2.lib.TestMutableMetrics



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764221/HADOOP-8437-005.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d6fa34e
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 30s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 8s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 10s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 6m 58s Tests failed in hadoop-common.     48m 1s   Reason Tests Failed unit tests hadoop.security.ssl.TestReloadingX509TrustManager Timed out tests org.apache.hadoop.ha.TestZKFailoverController   org.apache.hadoop.ha.TestZKFailoverControllerStress   org.apache.hadoop.metrics2.lib.TestMutableMetrics Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764221/HADOOP-8437-005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d6fa34e Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          The test failures are not related to the patch. All these tests passed in my local build.
          +1 LGTM for the latest patch, I will commit it tomorrow if no one objects.

          Show
          zxu zhihai xu added a comment - The test failures are not related to the patch. All these tests passed in my local build. +1 LGTM for the latest patch, I will commit it tomorrow if no one objects.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8553 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8553/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8553 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8553/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          zxu zhihai xu added a comment -

          Committed it to branch-2 and trunk! thanks Brahma Reddy Battula for the contribution! thanks Harsh J and Tsuyoshi Ozawa for additional review!

          Show
          zxu zhihai xu added a comment - Committed it to branch-2 and trunk! thanks Brahma Reddy Battula for the contribution! thanks Harsh J and Tsuyoshi Ozawa for additional review!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #478 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/478/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #478 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/478/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1208 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1208/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1208 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1208/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #470 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/470/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #470 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/470/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #444 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/444/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #444 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/444/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2384 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2384/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2384 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2384/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2413 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2413/)
          HADOOP-8437. getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2413 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2413/ ) HADOOP-8437 . getLocalPathForWrite should throw IOException for invalid (zxu: rev fd026f535cc09e99a7d4d5d2a8c13eabe8865315) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          thanks zhihai xu for review and commit, and others for comments..

          Show
          brahmareddy Brahma Reddy Battula added a comment - thanks zhihai xu for review and commit, and others for comments..

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development