Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8767

RawLocalFileSystem.listStatus() returns null for UNIX pipefile

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.2, 2.6.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Calling FileSystem.listStatus() on a UNIX pipe file returns null instead of the file. The bug breaks Hive when Hive loads data from UNIX pipe file.

      1. HDFS-8767.003.patch
        4 kB
        Haohui Mai
      2. HDFS-8767.004.patch
        5 kB
        Kanaka Kumar Avvaru
      3. HDFS-8767-00.patch
        2 kB
        Kanaka Kumar Avvaru
      4. HDFS-8767-01.patch
        4 kB
        Kanaka Kumar Avvaru
      5. HDFS-8767-02.patch
        5 kB
        Kanaka Kumar Avvaru
      6. HDFS-8767-branch-2.6.patch
        5 kB
        Akira Ajisaka

        Issue Links

          Activity

          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          As per java.io.File documentation, file.isFile() & file.isDirectory() return false and file.list() return null for non regular files like pipes.
          I this case RawLocalFileSystem.listStatus() returning null. So, we can check if the file is of type other and return the status

          Attached patch for fix

          Show
          kanaka Kanaka Kumar Avvaru added a comment - As per java.io.File documentation, file.isFile() & file.isDirectory() return false and file.list() return null for non regular files like pipes. I this case RawLocalFileSystem.listStatus() returning null. So, we can check if the file is of type other and return the status Attached patch for fix
          Hide
          stevel@apache.org Steve Loughran added a comment -

          this'll need a test for unix which at least downgrades on windows

          Show
          stevel@apache.org Steve Loughran added a comment - this'll need a test for unix which at least downgrades on windows
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 6s There were no new javac warning messages.
          +1 javadoc 10m 2s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 7s The applied patch generated 1 new checkstyle issues (total was 21, now 21).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 22m 4s Tests failed in hadoop-common.
              62m 44s  



          Reason Tests
          Failed unit tests hadoop.crypto.key.TestValueQueue



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745231/HDFS-8767-00.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4084eaf
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11698/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11698/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11698/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11698/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 6s There were no new javac warning messages. +1 javadoc 10m 2s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 7s The applied patch generated 1 new checkstyle issues (total was 21, now 21). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 4s Tests failed in hadoop-common.     62m 44s   Reason Tests Failed unit tests hadoop.crypto.key.TestValueQueue Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745231/HDFS-8767-00.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4084eaf checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11698/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11698/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11698/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11698/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Updated patch with the test case on UNIX based systems as per Steve Loughran comment

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Updated patch with the test case on UNIX based systems as per Steve Loughran comment
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 26s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 35s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 7s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 4s Tests passed in hadoop-common.
              61m 3s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745256/HDFS-8767-01.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4084eaf
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11700/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11700/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11700/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 26s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 7s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 4s Tests passed in hadoop-common.     61m 3s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745256/HDFS-8767-01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4084eaf hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11700/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11700/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11700/console This message was automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          It looks like that a cleaner approach is to call list() only when the file is a directory.

          Show
          wheat9 Haohui Mai added a comment - It looks like that a cleaner approach is to call list() only when the file is a directory.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for review Haohui Mai, I have updated patch as per your suggestion. Please review.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for review Haohui Mai , I have updated patch as per your suggestion. Please review.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 16s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 37s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 2s The applied patch generated 2 new checkstyle issues (total was 22, now 23).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 19s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 1s Tests passed in hadoop-common.
              60m 36s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745436/HDFS-8767-02.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / edcaae4
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11712/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11712/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11712/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11712/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 16s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 37s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 2s The applied patch generated 2 new checkstyle issues (total was 22, now 23). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 19s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 1s Tests passed in hadoop-common.     60m 36s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745436/HDFS-8767-02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / edcaae4 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11712/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11712/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11712/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11712/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Expected , Check style issues can be ignored

           /hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:474: Line is longer than 80 characters (found 82).

          – Fixing this will reduce the readability

          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:488:32: '{' is followed by whitespace.

          --The format looks fine in eclipse. Fixing this will reduce the readability

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Expected , Check style issues can be ignored /hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:474: Line is longer than 80 characters (found 82). – Fixing this will reduce the readability ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:488:32: '{' is followed by whitespace. --The format looks fine in eclipse. Fixing this will reduce the readability
          Hide
          wheat9 Haohui Mai added a comment -

          Thanks for the work. The fix looks good.

          --The format looks fine in eclipse. Fixing this will reduce the readability

          Readability is subjective. It might make more sense to fix it to avoid the checkstyle warnings.

          +  @Test
          +  public void testFileStatusPipeFile() throws Exception {
          +    Assume.assumeTrue(SystemUtils.IS_OS_UNIX);
          +    String path = TEST_ROOT_DIR + "/testfifofile";
          +    new File(path).delete();
          +    File fifoFile = new File(path);
          +    fifoFile.getParentFile().mkdirs();
          +    String fullPath = fifoFile.getAbsolutePath();
          +    Process process = Runtime.getRuntime().exec("mkfifo " + fullPath);
          +    process.waitFor();
          +
          +    String input = org.apache.commons.io.IOUtils.toString(process
          +        .getInputStream());
          +    String errors = org.apache.commons.io.IOUtils.toString(process
          +        .getErrorStream());
          +    assertTrue("Expected empty but got " + input, "".equals(input));
          +    assertTrue("Expected empty but got " + errors, "".equals(errors));
          +
          +    fifoFile = new File(fullPath);
          +    assertTrue("FIFO file should present", fifoFile.exists());
          +    assertFalse(fifoFile.isFile());
          +    assertFalse(fifoFile.isDirectory());
          +
          +    Path fsPath = new Path(path);
          +    FileSystem fs = fileSys.getRawFileSystem();
          +    assertTrue(fs.exists(fsPath));
          +    assertNotNull(fs.listStatus(fsPath));
          +    fifoFile.delete();
          +  }
           }
          

          To me it seems that it makes more sense to test it through mockito instead of creating a real pipe file. I'll upload a patch later to demonstrate the proposed approach.

          Show
          wheat9 Haohui Mai added a comment - Thanks for the work. The fix looks good. --The format looks fine in eclipse. Fixing this will reduce the readability Readability is subjective. It might make more sense to fix it to avoid the checkstyle warnings. + @Test + public void testFileStatusPipeFile() throws Exception { + Assume.assumeTrue(SystemUtils.IS_OS_UNIX); + String path = TEST_ROOT_DIR + "/testfifofile" ; + new File(path).delete(); + File fifoFile = new File(path); + fifoFile.getParentFile().mkdirs(); + String fullPath = fifoFile.getAbsolutePath(); + Process process = Runtime .getRuntime().exec( "mkfifo " + fullPath); + process.waitFor(); + + String input = org.apache.commons.io.IOUtils.toString(process + .getInputStream()); + String errors = org.apache.commons.io.IOUtils.toString(process + .getErrorStream()); + assertTrue( "Expected empty but got " + input, "".equals(input)); + assertTrue( "Expected empty but got " + errors, "".equals(errors)); + + fifoFile = new File(fullPath); + assertTrue( "FIFO file should present" , fifoFile.exists()); + assertFalse(fifoFile.isFile()); + assertFalse(fifoFile.isDirectory()); + + Path fsPath = new Path(path); + FileSystem fs = fileSys.getRawFileSystem(); + assertTrue(fs.exists(fsPath)); + assertNotNull(fs.listStatus(fsPath)); + fifoFile.delete(); + } } To me it seems that it makes more sense to test it through mockito instead of creating a real pipe file. I'll upload a patch later to demonstrate the proposed approach.
          Hide
          wheat9 Haohui Mai added a comment -

          The v3 patch is mostly identical to the v2 patch, except that it changes the test to use mockito so that it no longer depends on the mkfifo command in the test environment. Kanaka Kumar Avvaru, does it look good to you?

          Show
          wheat9 Haohui Mai added a comment - The v3 patch is mostly identical to the v2 patch, except that it changes the test to use mockito so that it no longer depends on the mkfifo command in the test environment. Kanaka Kumar Avvaru , does it look good to you?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 3s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 43s There were no new javac warning messages.
          +1 javadoc 10m 55s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 18s The applied patch generated 1 new checkstyle issues (total was 21, now 21).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 25s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 2m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 24m 1s Tests passed in hadoop-common.
              68m 35s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745529/HDFS-8767.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 3ec0a04
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11721/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11721/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11721/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11721/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 3s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 43s There were no new javac warning messages. +1 javadoc 10m 55s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 18s The applied patch generated 1 new checkstyle issues (total was 21, now 21). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 25s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 2m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 24m 1s Tests passed in hadoop-common.     68m 35s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745529/HDFS-8767.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 3ec0a04 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11721/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11721/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11721/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11721/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Looks fine for me. Thanks for updating the test Haohui Mai. I will post patch for pending check-style.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Looks fine for me. Thanks for updating the test Haohui Mai . I will post patch for pending check-style.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Updated patch.. Compared to v3, v4 is identical but just a try to fix check-style error reported in last build

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Updated patch.. Compared to v3, v4 is identical but just a try to fix check-style error reported in last build
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 27s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 35s There were no new javac warning messages.
          +1 javadoc 9m 35s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 3s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 20s Tests passed in hadoop-common.
              61m 13s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745652/HDFS-8767.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1ba2986
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11729/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11729/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11729/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 javadoc 9m 35s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 3s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 20s Tests passed in hadoop-common.     61m 13s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745652/HDFS-8767.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1ba2986 hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11729/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11729/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11729/console This message was automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          +1. I'll commit it shortly.

          Show
          wheat9 Haohui Mai added a comment - +1. I'll commit it shortly.
          Hide
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk, branch-2 and branch-2.7. Thanks Kanaka Kumar Avvaru for the contribution.

          Show
          wheat9 Haohui Mai added a comment - I've committed the patch to trunk, branch-2 and branch-2.7. Thanks Kanaka Kumar Avvaru for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8175 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8175/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8175 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8175/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #259 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/259/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #259 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/259/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #989 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/989/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #989 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/989/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/256/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/256/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2186/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2186 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2186/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #248 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/248/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #248 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/248/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2205 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2205/)
          HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2205 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2205/ ) HDFS-8767 . RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. (wheat9: rev d96bbe152cf536304208f2e8f35deb3b2aa91d2b) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          cnauroth Chris Nauroth added a comment -

          This patch introduced a test failure on Windows, which has now been fixed in HADOOP-12438.

          Show
          cnauroth Chris Nauroth added a comment - This patch introduced a test failure on Windows, which has now been fixed in HADOOP-12438 .
          Hide
          sjlee0 Sangjin Lee added a comment -

          Does this issue exist in 2.6.x? Should this be backported to branch-2.6?

          Show
          sjlee0 Sangjin Lee added a comment - Does this issue exist in 2.6.x? Should this be backported to branch-2.6?
          Hide
          djp Junping Du added a comment -

          Hi Haohui Mai and Chris Nauroth, does this fix should land on branch-2.6 also? Thanks!

          Show
          djp Junping Du added a comment - Hi Haohui Mai and Chris Nauroth , does this fix should land on branch-2.6 also? Thanks!
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for merging to branch-2.6. It looks like it will be a clean merge.

          Show
          cnauroth Chris Nauroth added a comment - +1 for merging to branch-2.6. It looks like it will be a clean merge.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Cherry-picked this to branch-2.6. I ran the regression test in the patch and it passed. Attaching the diff.
          Thanks Junping Du and Chris Nauroth!

          Show
          ajisakaa Akira Ajisaka added a comment - Cherry-picked this to branch-2.6. I ran the regression test in the patch and it passed. Attaching the diff. Thanks Junping Du and Chris Nauroth !
          Hide
          djp Junping Du added a comment -

          Thank u, Akira Ajisaka!

          Show
          djp Junping Du added a comment - Thank u, Akira Ajisaka !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9108 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9108/)
          Add missing CHANGES.txt entry for HDFS-8767. (junping_du: rev e50dd19a9c816c81c64fff8b958f4036dcb98822)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-yarn-project/.CHANGES.txt.swp
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9108 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9108/ ) Add missing CHANGES.txt entry for HDFS-8767 . (junping_du: rev e50dd19a9c816c81c64fff8b958f4036dcb98822) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-yarn-project/.CHANGES.txt.swp

            People

            • Assignee:
              kanaka Kanaka Kumar Avvaru
              Reporter:
              wheat9 Haohui Mai
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development