Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5684

Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as VisibleForTesting

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      o.a.h.fs.viewfs.ViewFileSystem provides the following method

        public MountPoint[] getMountPoints()
      

      However MountPoint has no public methods making getMountPoints() useless.

      1. HDFS-5684.01.patch
        3 kB
        Manoj Govindassamy
      2. HDFS-5684.02.patch
        3 kB
        Manoj Govindassamy

        Activity

        Hide
        manojg Manoj Govindassamy added a comment -

        Attaching v01 patch to expose src and targets from ViewFileSystem#MountPoint. Updated a test to make use of these methods. These public methods are more useful only if there are users from other packages.

        Show
        manojg Manoj Govindassamy added a comment - Attaching v01 patch to expose src and targets from ViewFileSystem#MountPoint . Updated a test to make use of these methods. These public methods are more useful only if there are users from other packages.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for working on this Manoj, this looks good overall, a few comments:

        • Is this a method that we want to expose to end user applications? It's possible this was just done for test purposes, which is why it's package-private. If so, we should just add some @VisibleForTesting annotations.
        • If this is supposed to be user-accessible, then we should consider moving it to a separate VfsAdmin class, the way we have HdfsAdmin for special HDFS APIs. We shouldn't make our users downcast to access what's meant to be a public API.
        • In the new test, I'd prefer if we used SLF4J rather than commons logging, SLF4J is our preferred logging library.
        Show
        andrew.wang Andrew Wang added a comment - Thanks for working on this Manoj, this looks good overall, a few comments: Is this a method that we want to expose to end user applications? It's possible this was just done for test purposes, which is why it's package-private. If so, we should just add some @VisibleForTesting annotations. If this is supposed to be user-accessible, then we should consider moving it to a separate VfsAdmin class, the way we have HdfsAdmin for special HDFS APIs. We shouldn't make our users downcast to access what's meant to be a public API. In the new test, I'd prefer if we used SLF4J rather than commons logging, SLF4J is our preferred logging library.
        Hide
        manojg Manoj Govindassamy added a comment - - edited
        • For now its used by tests only. We already expose org.apache.hadoop.fs.viewfs.ConfigUtil to client apps to programmatically add Mount points to the conf. May be these clients would be interested to look at all the existing mount points in the ViewFileSystem and it make sense to have a separate ViewFsAdmin. Even without ViewFsAdmin, client apps can now cast the FileSystem to ViewFileSystem and access MountPoints if needed. Without src/target methods being public, client apps will not be able to see them. But, I think having a separate Admin class is much cleaner than they having to cast and get internal details.
        • Updated the test to make use of SL4J instead of commons logging.

        Attached v02 patch. Please take a look.

        Show
        manojg Manoj Govindassamy added a comment - - edited For now its used by tests only. We already expose org.apache.hadoop.fs.viewfs.ConfigUtil to client apps to programmatically add Mount points to the conf. May be these clients would be interested to look at all the existing mount points in the ViewFileSystem and it make sense to have a separate ViewFsAdmin. Even without ViewFsAdmin, client apps can now cast the FileSystem to ViewFileSystem and access MountPoints if needed. Without src/target methods being public, client apps will not be able to see them. But, I think having a separate Admin class is much cleaner than they having to cast and get internal details. Updated the test to make use of SL4J instead of commons logging. Attached v02 patch. Please take a look.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 12s trunk passed
        +1 compile 8m 37s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 32s trunk passed
        +1 javadoc 0m 46s trunk passed
        +1 mvninstall 0m 44s the patch passed
        +1 compile 8m 27s the patch passed
        +1 javac 8m 27s the patch passed
        +1 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 0 new + 111 unchanged - 2 fixed = 111 total (was 113)
        +1 mvnsite 1m 4s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 39s the patch passed
        +1 javadoc 0m 45s the patch passed
        -1 unit 9m 53s hadoop-common in the patch failed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        45m 25s



        Reason Tests
        Failed junit tests hadoop.ha.TestZKFailoverController



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-5684
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835203/HDFS-5684.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 05476c55435c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 86c735b
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17281/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17281/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 12s trunk passed +1 compile 8m 37s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 32s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 44s the patch passed +1 compile 8m 27s the patch passed +1 javac 8m 27s the patch passed +1 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 0 new + 111 unchanged - 2 fixed = 111 total (was 113) +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 39s the patch passed +1 javadoc 0m 45s the patch passed -1 unit 9m 53s hadoop-common in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 45m 25s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-5684 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835203/HDFS-5684.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 05476c55435c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 86c735b Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17281/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17281/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        +1 LGTM, that failure looks like a flake. Will commit shortly.

        Show
        andrew.wang Andrew Wang added a comment - +1 LGTM, that failure looks like a flake. Will commit shortly.
        Hide
        andrew.wang Andrew Wang added a comment -

        Committed to trunk, thanks Manoj for working on this!

        Show
        andrew.wang Andrew Wang added a comment - Committed to trunk, thanks Manoj for working on this!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10678 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10678/)
        HDFS-5684. Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as (wang: rev 084bdab156fd08b61484c992e25ce964b46568e8)

        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
        • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10678 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10678/ ) HDFS-5684 . Annotate o.a.h.fs.viewfs.ViewFileSystem.MountPoint as (wang: rev 084bdab156fd08b61484c992e25ce964b46568e8) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java

          People

          • Assignee:
            manojg Manoj Govindassamy
            Reporter:
            kturner Keith Turner
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development