Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7165

listLocatedStatus(path, filter) is not redefined in FilterFs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      listLocatedStatus(path, filter) is not redefined in FilterFs. So if a job client uses a FilterFs to talk to NameNode, it does not trigger the bulk location optimization.

      1. locatedStatusFilter.patch
        1 kB
        Hairong Kuang
      2. HADOOP-7165.patch
        2 kB
        Harsh J

        Activity

        Hide
        hairong Hairong Kuang added a comment -

        Here is a straightforward fix.

        Show
        hairong Hairong Kuang added a comment - Here is a straightforward fix.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        qwertymaniac Harsh J added a comment -

        Rebased patch for trunk.

        Show
        qwertymaniac Harsh J added a comment - Rebased patch for trunk.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 1s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 44s There were no new javac warning messages.
        +1 javadoc 9m 56s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 5s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 44s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 24m 48s Tests passed in hadoop-common.
            62m 52s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12546154/HADOOP-7165.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 1s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 56s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 5s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 44s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 24m 48s Tests passed in hadoop-common.     62m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12546154/HADOOP-7165.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6381/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 4s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 35s There were no new javac warning messages.
        +1 javadoc 9m 45s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 5s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 35s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 22m 30s Tests passed in hadoop-common.
            60m 15s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12546154/HADOOP-7165.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 35s There were no new javac warning messages. +1 javadoc 9m 45s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 5s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 30s Tests passed in hadoop-common.     60m 15s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12546154/HADOOP-7165.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6383/console This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        +1 lgtm. Committing this.

        Show
        jlowe Jason Lowe added a comment - +1 lgtm. Committing this.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks to Hairong Kuang for the original contribution, Harsh J for upmerging the patch, and Tsz Wo Nicholas Sze for additional review! I committed this to trunk and branch-2.

        Show
        jlowe Jason Lowe added a comment - Thanks to Hairong Kuang for the original contribution, Harsh J for upmerging the patch, and Tsz Wo Nicholas Sze for additional review! I committed this to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7779 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7779/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7779 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7779/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #191 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/191/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #191 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/191/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #922 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/922/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #922 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/922/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2120 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2120/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2120 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2120/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/180/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/180/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/190/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/190/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2138 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2138/)
        HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2138 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2138/ ) HADOOP-7165 . listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang (jlowe: rev c4bd2cddede779acfd3c5cdff16bf552068dfb64) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java

          People

          • Assignee:
            hairong Hairong Kuang
            Reporter:
            hairong Hairong Kuang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development