Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10914

Move remnants of oah.hdfs.client to hadoop-hdfs-client

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      The remaining classes in the org.apache.hadoop.hdfs.client package have been moved from hadoop-hdfs to hadoop-hdfs-client.

      Description

      Some remaining classes in the oah.hdfs.client package are still in hadoop-hdfs rather than hadoop-hdfs-client.

      This broke a client that depended on hadoop-client for HdfsAdmin. hadoop-client now pulls in hadoop-hdfs-client rather than hadoop-hdfs, meaning it lost access to HdfsAdmin.

      1. hdfs-10914.001.patch
        53 kB
        Andrew Wang
      2. hdfs-10914.002.patch
        54 kB
        Andrew Wang

        Activity

        Hide
        andrew.wang Andrew Wang added a comment -

        Patch attached. I encourage reviewers to look at the "git diff -M" to see changes, listed here:

        • Switched HdfsUtils to SLF4J to avoid a apache commons logging dependency
        • Switched to just closing the fs at the end of HdfsUtils, since it'll still log in DistributedFileSystem.
        • Removed javadoc on DFSAdmin, since it depends on hadoop-hdfs
        • Added imports to CreateEncryptionZoneFlag to fix javadoc links
        Show
        andrew.wang Andrew Wang added a comment - Patch attached. I encourage reviewers to look at the "git diff -M" to see changes, listed here: Switched HdfsUtils to SLF4J to avoid a apache commons logging dependency Switched to just closing the fs at the end of HdfsUtils, since it'll still log in DistributedFileSystem. Removed javadoc on DFSAdmin, since it depends on hadoop-hdfs Added imports to CreateEncryptionZoneFlag to fix javadoc links
        Hide
        andrew.wang Andrew Wang added a comment -

        FWIW, verified that HdfsAdmin was present in the new JAR too:

        -> % jar -tf ./hadoop-hdfs-project/hadoop-hdfs-client/target/hadoop-hdfs-client-3.0.0-alpha2-SNAPSHOT.jar | grep HdfsAdmin
        org/apache/hadoop/hdfs/client/HdfsAdmin.class
        
        Show
        andrew.wang Andrew Wang added a comment - FWIW, verified that HdfsAdmin was present in the new JAR too: -> % jar -tf ./hadoop-hdfs-project/hadoop-hdfs-client/target/hadoop-hdfs-client-3.0.0-alpha2-SNAPSHOT.jar | grep HdfsAdmin org/apache/hadoop/hdfs/client/HdfsAdmin.class
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 7s Maven dependency ordering for branch
        +1 mvninstall 8m 26s trunk passed
        +1 compile 1m 46s trunk passed
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 1m 38s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 21s trunk passed
        +1 javadoc 1m 15s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 18s the patch passed
        +1 compile 1m 21s the patch passed
        +1 javac 1m 21s the patch passed
        -0 checkstyle 0m 28s hadoop-hdfs-project: The patch generated 3 new + 4 unchanged - 1 fixed = 7 total (was 5)
        +1 mvnsite 1m 21s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        -1 whitespace 0m 0s The patch has 24 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 3m 21s the patch passed
        +1 javadoc 1m 14s the patch passed
        +1 unit 0m 57s hadoop-hdfs-client in the patch passed.
        +1 unit 59m 36s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        89m 34s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10914
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830576/hdfs-10914.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1e87ec4824bc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1831be8
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16886/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16886/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16886/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16886/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 8m 26s trunk passed +1 compile 1m 46s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 38s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 21s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 18s the patch passed +1 compile 1m 21s the patch passed +1 javac 1m 21s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project: The patch generated 3 new + 4 unchanged - 1 fixed = 7 total (was 5) +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 20s the patch passed -1 whitespace 0m 0s The patch has 24 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 3m 21s the patch passed +1 javadoc 1m 14s the patch passed +1 unit 0m 57s hadoop-hdfs-client in the patch passed. +1 unit 59m 36s hadoop-hdfs in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 89m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10914 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830576/hdfs-10914.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1e87ec4824bc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1831be8 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16886/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16886/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16886/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16886/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        New rev attached. Apparently checkstyle doesn't like those new imports, and does like package-info.java files.

        Whitespace is an existing issue, since I'm just renaming. I can do --whitespace=fix at commit time though.

        Show
        andrew.wang Andrew Wang added a comment - New rev attached. Apparently checkstyle doesn't like those new imports, and does like package-info.java files. Whitespace is an existing issue, since I'm just renaming. I can do --whitespace=fix at commit time though.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        The patch is mostly moving code around. +1 pending jenkins

        Thanks Andrew Wang

        Show
        eddyxu Lei (Eddy) Xu added a comment - The patch is mostly moving code around. +1 pending jenkins Thanks Andrew Wang
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 25s Maven dependency ordering for branch
        +1 mvninstall 7m 4s trunk passed
        +1 compile 1m 27s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 26s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 3m 8s trunk passed
        +1 javadoc 1m 15s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 16s the patch passed
        +1 compile 1m 22s the patch passed
        +1 javac 1m 22s the patch passed
        +1 checkstyle 0m 27s hadoop-hdfs-project: The patch generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5)
        +1 mvnsite 1m 21s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        -1 whitespace 0m 0s The patch has 24 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 3m 37s the patch passed
        +1 javadoc 1m 18s the patch passed
        +1 unit 1m 3s hadoop-hdfs-client in the patch passed.
        -1 unit 77m 21s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 27s The patch does not generate ASF License warnings.
        106m 1s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.mover.TestStorageMover



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10914
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830592/hdfs-10914.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9823a5e74d5e 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2acfb1e
        Default Java 1.8.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16890/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16890/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16890/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16890/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 7m 4s trunk passed +1 compile 1m 27s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 3m 8s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 16s the patch passed +1 compile 1m 22s the patch passed +1 javac 1m 22s the patch passed +1 checkstyle 0m 27s hadoop-hdfs-project: The patch generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 20s the patch passed -1 whitespace 0m 0s The patch has 24 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 3m 37s the patch passed +1 javadoc 1m 18s the patch passed +1 unit 1m 3s hadoop-hdfs-client in the patch passed. -1 unit 77m 21s hadoop-hdfs in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 106m 1s Reason Tests Failed junit tests hadoop.hdfs.server.mover.TestStorageMover Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10914 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830592/hdfs-10914.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9823a5e74d5e 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2acfb1e Default Java 1.8.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16890/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16890/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16890/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16890/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for reviewing Eddy, committed to trunk, branch-2, branch-2.8

        Show
        andrew.wang Andrew Wang added a comment - Thanks for reviewing Eddy, committed to trunk, branch-2, branch-2.8
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10509 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10509/)
        HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client. (wang: rev 92e5e9159850c01635091ea6ded0d8ee76691a9a)

        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/CreateEncryptionZoneFlag.java
        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/package-info.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java
        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/package-info.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/CreateEncryptionZoneFlag.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10509 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10509/ ) HDFS-10914 . Move remnants of oah.hdfs.client to hadoop-hdfs-client. (wang: rev 92e5e9159850c01635091ea6ded0d8ee76691a9a) (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/CreateEncryptionZoneFlag.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/package-info.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/package-info.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/CreateEncryptionZoneFlag.java

          People

          • Assignee:
            andrew.wang Andrew Wang
            Reporter:
            andrew.wang Andrew Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development