Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8941

DistributedFileSystem listCorruptFileBlocks API should resolve relative path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:

      Description

      Presently DFS#listCorruptFileBlocks(path) API is not resolving the given path relative to the workingDir. This jira is to discuss and provide the implementation of the same.

      1. HDFS-8941-00.patch
        5 kB
        Rakesh R
      2. HDFS-8941-01.patch
        5 kB
        Rakesh R
      3. HDFS-8941-02.patch
        5 kB
        Rakesh R
      4. HDFS-8941-03.patch
        5 kB
        Rakesh R
      5. HDFS-8941-04.patch
        5 kB
        Rakesh R

        Activity

        Hide
        rakeshr Rakesh R added a comment -

        Thank you Andrew Wang for reviewing and committing the patch!

        Show
        rakeshr Rakesh R added a comment - Thank you Andrew Wang for reviewing and committing the patch!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1243 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1243/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1243 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1243/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/)
        HDFS-8941. DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/ ) HDFS-8941 . DistributedFileSystem listCorruptFileBlocks API should (wang: rev c32614f410fb62a7179abfefbab42a05415a3066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java
        Hide
        andrew.wang Andrew Wang added a comment -

        Committed to trunk and branch-2, thanks for the contribution Rakesh!

        Show
        andrew.wang Andrew Wang added a comment - Committed to trunk and branch-2, thanks for the contribution Rakesh!
        Hide
        andrew.wang Andrew Wang added a comment -

        +1 LGTM will commit shortly

        Show
        andrew.wang Andrew Wang added a comment - +1 LGTM will commit shortly
        Hide
        rakeshr Rakesh R added a comment -

        Please ignore test case failure and checkstyle warning, its not related to the patch.

        Show
        rakeshr Rakesh R added a comment - Please ignore test case failure and checkstyle warning, its not related to the patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 20m 12s Pre-patch trunk has 748 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 9s There were no new javac warning messages.
        +1 javadoc 10m 22s There were no new javadoc warning messages.
        -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
        -1 checkstyle 2m 57s The applied patch generated 1 new checkstyle issues (total was 21, now 21).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 4m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 20s Pre-build of native portion
        -1 hdfs tests 90m 10s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 31s Tests passed in hadoop-hdfs-client.
            142m 59s  



        Reason Tests
        Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765509/HDFS-8941-04.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1107bd3
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12856/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12856/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 12s Pre-patch trunk has 748 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 9s There were no new javac warning messages. +1 javadoc 10m 22s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. -1 checkstyle 2m 57s The applied patch generated 1 new checkstyle issues (total was 21, now 21). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 20s Pre-build of native portion -1 hdfs tests 90m 10s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 31s Tests passed in hadoop-hdfs-client.     142m 59s   Reason Tests Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765509/HDFS-8941-04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1107bd3 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12856/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12856/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12856/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 30m 14s Pre-patch trunk has 748 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 11m 26s There were no new javac warning messages.
        +1 javadoc 15m 47s There were no new javadoc warning messages.
        -1 release audit 0m 36s The applied patch generated 1 release audit warnings.
        -1 checkstyle 4m 21s The applied patch generated 1 new checkstyle issues (total was 21, now 21).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 2m 27s mvn install still works.
        +1 eclipse:eclipse 0m 48s The patch built with eclipse:eclipse.
        +1 findbugs 6m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 4m 43s Pre-build of native portion
        -1 hdfs tests 112m 53s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 42s Tests passed in hadoop-hdfs-client.
            190m 59s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestClusterId
        Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot
          org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
          org.apache.hadoop.hdfs.TestEncryptedTransfer
          org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765509/HDFS-8941-04.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 35affec
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12849/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12849/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 30m 14s Pre-patch trunk has 748 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 11m 26s There were no new javac warning messages. +1 javadoc 15m 47s There were no new javadoc warning messages. -1 release audit 0m 36s The applied patch generated 1 release audit warnings. -1 checkstyle 4m 21s The applied patch generated 1 new checkstyle issues (total was 21, now 21). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 2m 27s mvn install still works. +1 eclipse:eclipse 0m 48s The patch built with eclipse:eclipse. +1 findbugs 6m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 4m 43s Pre-build of native portion -1 hdfs tests 112m 53s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 42s Tests passed in hadoop-hdfs-client.     190m 59s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestClusterId Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot   org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   org.apache.hadoop.hdfs.TestEncryptedTransfer   org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765509/HDFS-8941-04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 35affec Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12849/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12849/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12849/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Thank you Andrew Wang, I've re-based the patch in latest trunk code and attached the same. Please take a look at it again!

        Show
        rakeshr Rakesh R added a comment - Thank you Andrew Wang , I've re-based the patch in latest trunk code and attached the same. Please take a look at it again!
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for working on this Rakesh! Patch LGTM, but needs a small rebase. Will commit after it comes back from Jenkins.

        Show
        andrew.wang Andrew Wang added a comment - Thanks for working on this Rakesh! Patch LGTM, but needs a small rebase. Will commit after it comes back from Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 20m 55s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 11s There were no new javac warning messages.
        +1 javadoc 10m 17s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 56s The applied patch generated 1 new checkstyle issues (total was 45, now 45).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        -1 findbugs 0m 56s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken.
        -1 findbugs 3m 30s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 29s Pre-build of native portion
        -1 hdfs tests 65m 0s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.
            117m 24s  



        Reason Tests
        FindBugs module:hadoop-hdfs-client
        Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.server.blockmanagement.TestNodeCount
        Timed out tests org.apache.hadoop.hdfs.TestDecommission
          org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          org.apache.hadoop.hdfs.TestDFSOutputStream



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764199/HDFS-8941-03.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 151fca5
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-client.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12738/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12738/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 55s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 11s There were no new javac warning messages. +1 javadoc 10m 17s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 56s The applied patch generated 1 new checkstyle issues (total was 45, now 45). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 0m 56s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken. -1 findbugs 3m 30s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 29s Pre-build of native portion -1 hdfs tests 65m 0s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.     117m 24s   Reason Tests FindBugs module:hadoop-hdfs-client Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.blockmanagement.TestNodeCount Timed out tests org.apache.hadoop.hdfs.TestDecommission   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   org.apache.hadoop.hdfs.TestDFSOutputStream Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764199/HDFS-8941-03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 151fca5 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-client.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12738/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12738/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12738/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Rebased the patch and attached to the jira, as HDFS-8740 has moved the DistributedFileSystem.java class to hadoop-hdfs-client module.

        Show
        rakeshr Rakesh R added a comment - Rebased the patch and attached to the jira, as HDFS-8740 has moved the DistributedFileSystem.java class to hadoop-hdfs-client module.
        Hide
        rakeshr Rakesh R added a comment -

        It seems the checkstyle warnings and hdfs tests failures are unrelated to the patch, please ignore it.

        Show
        rakeshr Rakesh R added a comment - It seems the checkstyle warnings and hdfs tests failures are unrelated to the patch, please ignore it.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 56s There were no new javac warning messages.
        +1 javadoc 10m 7s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 46, now 46).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 10s Pre-build of native portion
        -1 hdfs tests 133m 48s Tests failed in hadoop-hdfs.
            179m 19s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
          hadoop.hdfs.web.TestWebHDFSOAuth2
        Timed out tests org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          org.apache.hadoop.hdfs.server.mover.TestStorageMover



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12756146/HDFS-8941-02.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 2ffe2db
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12467/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12467/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12467/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12467/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 56s There were no new javac warning messages. +1 javadoc 10m 7s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 46, now 46). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 133m 48s Tests failed in hadoop-hdfs.     179m 19s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.web.TestWebHDFSOAuth2 Timed out tests org.apache.hadoop.hdfs.server.namenode.TestBackupNode   org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   org.apache.hadoop.hdfs.server.mover.TestStorageMover Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756146/HDFS-8941-02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2ffe2db checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12467/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12467/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12467/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12467/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Attached another patch based on the latest trunk code base.

        Show
        rakeshr Rakesh R added a comment - Attached another patch based on the latest trunk code base.
        Hide
        rakeshr Rakesh R added a comment -

        Please ignore the checkstyle warnings and hdfs tests failures, those are unrelated to the proposed patch.

        Show
        rakeshr Rakesh R added a comment - Please ignore the checkstyle warnings and hdfs tests failures, those are unrelated to the proposed patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 23m 28s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 10m 9s There were no new javac warning messages.
        +1 javadoc 11m 48s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 31s The applied patch generated 1 new checkstyle issues (total was 46, now 46).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 18s Pre-build of native portion
        -1 hdfs tests 187m 59s Tests failed in hadoop-hdfs.
            243m 21s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.server.namenode.TestFileTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753912/HDFS-8941-01.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 09c64ba
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12271/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12271/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12271/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12271/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 28s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 10m 9s There were no new javac warning messages. +1 javadoc 11m 48s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 31s The applied patch generated 1 new checkstyle issues (total was 46, now 46). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 18s Pre-build of native portion -1 hdfs tests 187m 59s Tests failed in hadoop-hdfs.     243m 21s   Reason Tests Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753912/HDFS-8941-01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 09c64ba checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12271/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12271/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12271/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12271/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks a lot Andrew Wang for the comments and the explanation. Attached another patch addressing the same.

        Show
        rakeshr Rakesh R added a comment - Thanks a lot Andrew Wang for the comments and the explanation. Attached another patch addressing the same.
        Hide
        andrew.wang Andrew Wang added a comment -

        Hi Rakesh, thanks for picking up this issue. One question, why do we cast to DFS in next()? Symlinks can be cross-filesystem, and other filesystems could implement this API. Most of the time you want to leave this as just a FileSystem without casting.

        Otherwise looks good though, thx again.

        Show
        andrew.wang Andrew Wang added a comment - Hi Rakesh, thanks for picking up this issue. One question, why do we cast to DFS in next()? Symlinks can be cross-filesystem, and other filesystems could implement this API. Most of the time you want to leave this as just a FileSystem without casting. Otherwise looks good though, thx again.
        Hide
        rakeshr Rakesh R added a comment -

        It seems the test case failures are not related to the patch. Let me try to trigger the jenkins again for a clean report!

        Show
        rakeshr Rakesh R added a comment - It seems the test case failures are not related to the patch. Let me try to trigger the jenkins again for a clean report!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 3s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 54s There were no new javac warning messages.
        +1 javadoc 10m 14s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 25s The applied patch generated 1 new checkstyle issues (total was 46, now 46).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 29s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 15s Pre-build of native portion
        -1 hdfs tests 163m 44s Tests failed in hadoop-hdfs.
            209m 40s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestLeaseRecovery2
          hadoop.hdfs.TestCrcCorruption



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12751879/HDFS-8941-00.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / eee0d45
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12111/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12111/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 3s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 14s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 25s The applied patch generated 1 new checkstyle issues (total was 46, now 46). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 15s Pre-build of native portion -1 hdfs tests 163m 44s Tests failed in hadoop-hdfs.     209m 40s   Reason Tests Failed unit tests hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.TestCrcCorruption Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751879/HDFS-8941-00.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12111/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12111/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12111/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Attached patch, where it resolves the path relative to the workingDir. Please review the scenario and the proposed fix. Thanks!

        Show
        rakeshr Rakesh R added a comment - Attached patch, where it resolves the path relative to the workingDir . Please review the scenario and the proposed fix. Thanks!

          People

          • Assignee:
            rakeshr Rakesh R
            Reporter:
            rakeshr Rakesh R
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development