Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11432

Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: federation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As of now, client needs to change "fs.defaultFS" to connect to specific name service when cluster is federated cluster which will be inconvenient for user.

      Snapshot

       hdfs dfsadmin -allowSnapshot hdfs://hacluster/Dir1/snapdir2
      allowSnapshot: FileSystem viewfs://ClusterX/ is not an HDFS file system
      Usage: hdfs dfsadmin [-allowSnapshot <snapshotDir>]
      

      Quota

      hdfs dfsadmin -setQuota 10 hdfs://hacluster/Bulkload
      setQuota: FileSystem viewfs://ClusterX/ is not an HDFS file system
      Usage: hdfs dfsadmin [-setQuota <quota> <dirname>...<dirname>]
      
      1. HDFS-11432.patch
        15 kB
        Brahma Reddy Battula
      2. HDFS-11432-002.patch
        15 kB
        Brahma Reddy Battula
      3. HDFS-11432-003.patch
        18 kB
        Brahma Reddy Battula
      4. HDFS-11432-004.patch
        18 kB
        Brahma Reddy Battula

        Activity

        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Basically all the path based commands should support fully qualified path
        will address the following as part of this jira..Bytheway HDFS-11177 address the storagepolicyadmin

        Command Having Path Command Description
        balancer No run a cluster balancing utility
        cacheadmin Yes configure the HDFS cache
        classpath No prints the class path needed to get the hadoop jar and the required libraries
        crypto Yes configure HDFS encryption zones
        datanode No run a DFS datanode
        debug Yes run a Debug Admin to execute HDFS debug commands
        dfsadmin Yes run a DFS admin client
        dfs Yes run a filesystem command on the file system
        diskbalancer No Distributes data evenly among disks on a given node
        envvars No display computed Hadoop environment variables
        erasurecode Yes run a HDFS ErasureCoding CLI
        fetchdt Yes a delegation token from the NameNode
        fsck Yes run a DFS filesystem checking utility
        getconf No get config values from configuration
        groups No get the groups which users belong to
        haadmin Yes run a DFS HA admin client
        jmxget NA get JMX exported values from NameNode or DataNode.
        journalnode NA run the DFS journalnode
        lsSnapshottableDir No list all snapshottable dirs owned by the current user
        mover Yes run a utility to move block replicas across storage types
        namenode No run the DFS namenode
        nfs3 No run an NFS version 3 gateway
        oev NA apply the offline edits viewer to an edits file
        oiv NA apply the offline fsimage viewer to an fsimage
        oiv_legacy NA apply the offline fsimage viewer to a legacy fsimage
        portmap NA run a portmap service
        secondarynamenode NA run the DFS secondary namenode
        snapshotDiff Yes diff two snapshots of a directory or diff the current directory contents with a snapshot
        storagepolicies Yes list/get/set block storage policies
        version No print the version
        zkfc NA run the ZK Failover Controller daemon
        Show
        brahmareddy Brahma Reddy Battula added a comment - Basically all the path based commands should support fully qualified path will address the following as part of this jira..Bytheway HDFS-11177 address the storagepolicyadmin Command Having Path Command Description balancer No run a cluster balancing utility cacheadmin Yes configure the HDFS cache classpath No prints the class path needed to get the hadoop jar and the required libraries crypto Yes configure HDFS encryption zones datanode No run a DFS datanode debug Yes run a Debug Admin to execute HDFS debug commands dfsadmin Yes run a DFS admin client dfs Yes run a filesystem command on the file system diskbalancer No Distributes data evenly among disks on a given node envvars No display computed Hadoop environment variables erasurecode Yes run a HDFS ErasureCoding CLI fetchdt Yes a delegation token from the NameNode fsck Yes run a DFS filesystem checking utility getconf No get config values from configuration groups No get the groups which users belong to haadmin Yes run a DFS HA admin client jmxget NA get JMX exported values from NameNode or DataNode. journalnode NA run the DFS journalnode lsSnapshottableDir No list all snapshottable dirs owned by the current user mover Yes run a utility to move block replicas across storage types namenode No run the DFS namenode nfs3 No run an NFS version 3 gateway oev NA apply the offline edits viewer to an edits file oiv NA apply the offline fsimage viewer to an fsimage oiv_legacy NA apply the offline fsimage viewer to a legacy fsimage portmap NA run a portmap service secondarynamenode NA run the DFS secondary namenode snapshotDiff Yes diff two snapshots of a directory or diff the current directory contents with a snapshot storagepolicies Yes list/get/set block storage policies version No print the version zkfc NA run the ZK Failover Controller daemon
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploading the initial patch..

        Addressed for quota/snapshot/cryptoadmin/cacheadmin which are having path as mandatory.

        Bytheway, as we know all commands will support "-fs" option(which will override the fs.defaultFS of client config),so user can dynamically provide(In this way commands doesn't have path also supported for dynamic nameserivce ).

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploading the initial patch.. Addressed for quota/snapshot/cryptoadmin/cacheadmin which are having path as mandatory. Bytheway, as we know all commands will support "-fs" option(which will override the fs.defaultFS of client config),so user can dynamically provide(In this way commands doesn't have path also supported for dynamic nameserivce ).
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 13s Maven dependency ordering for branch
        +1 mvninstall 12m 48s trunk passed
        +1 compile 13m 1s trunk passed
        +1 checkstyle 1m 53s trunk passed
        +1 mvnsite 1m 58s trunk passed
        +1 mvneclipse 0m 36s trunk passed
        +1 findbugs 3m 13s trunk passed
        +1 javadoc 1m 32s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 23s the patch passed
        +1 compile 11m 6s the patch passed
        +1 javac 11m 6s the patch passed
        +1 checkstyle 1m 54s the patch passed
        +1 mvnsite 1m 56s the patch passed
        +1 mvneclipse 0m 37s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 31s the patch passed
        +1 javadoc 1m 36s the patch passed
        -1 unit 8m 18s hadoop-common in the patch failed.
        -1 unit 73m 17s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 32s The patch does not generate ASF License warnings.
        141m 2s



        Reason Tests
        Failed junit tests hadoop.security.TestKDiag
          hadoop.hdfs.TestSnapshotCommands
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestErasureCodeBenchmarkThroughput



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11432
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854208/HDFS-11432.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 900bab817f3a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a207aa9
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18425/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18425/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18425/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 12m 48s trunk passed +1 compile 13m 1s trunk passed +1 checkstyle 1m 53s trunk passed +1 mvnsite 1m 58s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 3m 13s trunk passed +1 javadoc 1m 32s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 23s the patch passed +1 compile 11m 6s the patch passed +1 javac 11m 6s the patch passed +1 checkstyle 1m 54s the patch passed +1 mvnsite 1m 56s the patch passed +1 mvneclipse 0m 37s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 31s the patch passed +1 javadoc 1m 36s the patch passed -1 unit 8m 18s hadoop-common in the patch failed. -1 unit 73m 17s hadoop-hdfs in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 141m 2s Reason Tests Failed junit tests hadoop.security.TestKDiag   hadoop.hdfs.TestSnapshotCommands   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestErasureCodeBenchmarkThroughput Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11432 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854208/HDFS-11432.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 900bab817f3a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a207aa9 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18425/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18425/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18425/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the patch to address the testcase failure..
        ECadmin(which is newly added) addressed in HDFS-11426.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch to address the testcase failure.. ECadmin (which is newly added) addressed in HDFS-11426 .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 11m 54s trunk passed
        +1 compile 12m 32s trunk passed
        +1 checkstyle 1m 42s trunk passed
        +1 mvnsite 1m 48s trunk passed
        +1 mvneclipse 0m 32s trunk passed
        +1 findbugs 3m 3s trunk passed
        +1 javadoc 1m 25s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 15s the patch passed
        +1 compile 10m 4s the patch passed
        +1 javac 10m 4s the patch passed
        +1 checkstyle 1m 46s the patch passed
        +1 mvnsite 1m 50s the patch passed
        +1 mvneclipse 0m 39s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 16s the patch passed
        +1 javadoc 1m 24s the patch passed
        +1 unit 7m 41s hadoop-common in the patch passed.
        -1 unit 63m 33s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        126m 51s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
          hadoop.hdfs.web.TestWebHdfsTimeouts



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11432
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854350/HDFS-11432-002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9f94acf0f6cc 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132f758
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18434/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18434/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18434/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 11m 54s trunk passed +1 compile 12m 32s trunk passed +1 checkstyle 1m 42s trunk passed +1 mvnsite 1m 48s trunk passed +1 mvneclipse 0m 32s trunk passed +1 findbugs 3m 3s trunk passed +1 javadoc 1m 25s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 15s the patch passed +1 compile 10m 4s the patch passed +1 javac 10m 4s the patch passed +1 checkstyle 1m 46s the patch passed +1 mvnsite 1m 50s the patch passed +1 mvneclipse 0m 39s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 16s the patch passed +1 javadoc 1m 24s the patch passed +1 unit 7m 41s hadoop-common in the patch passed. -1 unit 63m 33s hadoop-hdfs in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 126m 51s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.web.TestWebHdfsTimeouts Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11432 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854350/HDFS-11432-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9f94acf0f6cc 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132f758 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18434/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18434/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18434/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Test failures are unrelated . HDFS-11454 raised to track TestUpdatePipelineWithSnapshots

        Show
        brahmareddy Brahma Reddy Battula added a comment - Test failures are unrelated . HDFS-11454 raised to track TestUpdatePipelineWithSnapshots
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        uploaded the patch to add for snapshotdiff command also.Kindly review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - uploaded the patch to add for snapshotdiff command also.Kindly review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 12m 40s trunk passed
        +1 compile 13m 21s trunk passed
        +1 checkstyle 1m 49s trunk passed
        +1 mvnsite 2m 4s trunk passed
        +1 mvneclipse 0m 43s trunk passed
        +1 findbugs 3m 26s trunk passed
        +1 javadoc 1m 35s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 30s the patch passed
        +1 compile 12m 18s the patch passed
        +1 javac 12m 18s the patch passed
        +1 checkstyle 1m 53s the patch passed
        +1 mvnsite 1m 55s the patch passed
        +1 mvneclipse 0m 38s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 29s the patch passed
        +1 javadoc 1m 34s the patch passed
        -1 unit 8m 15s hadoop-common in the patch failed.
        -1 unit 97m 38s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 37s The patch does not generate ASF License warnings.
        167m 23s



        Reason Tests
        Failed junit tests hadoop.security.TestRaceWhenRelogin
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.TestFileCorruption
          hadoop.hdfs.server.namenode.ha.TestHAAppend
        Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11432
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854661/HDFS-11432-003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e3ecb87478b8 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 120bef7
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18445/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18445/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18445/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18445/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 12m 40s trunk passed +1 compile 13m 21s trunk passed +1 checkstyle 1m 49s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 43s trunk passed +1 findbugs 3m 26s trunk passed +1 javadoc 1m 35s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 12m 18s the patch passed +1 javac 12m 18s the patch passed +1 checkstyle 1m 53s the patch passed +1 mvnsite 1m 55s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 29s the patch passed +1 javadoc 1m 34s the patch passed -1 unit 8m 15s hadoop-common in the patch failed. -1 unit 97m 38s hadoop-hdfs in the patch failed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 167m 23s Reason Tests Failed junit tests hadoop.security.TestRaceWhenRelogin   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.server.namenode.ha.TestHAAppend Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11432 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854661/HDFS-11432-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e3ecb87478b8 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 120bef7 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18445/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18445/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18445/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18445/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Test Failures are unrelated .

        Show
        brahmareddy Brahma Reddy Battula added a comment - Test Failures are unrelated .
        Hide
        vinayrpet Vinayakumar B added a comment -

        Looks almost Good.
        Just two nits.
        1. testSnapshotCommandsWithURI(), add timeout ~60sec.
        2. better to explain in comments why you using fresh config with different URI in below code.

        	Configuration config = new HdfsConfiguration();
               config.set("fs.defaultFS", "hdfs://127.0.0.1:1024");
               String path = fs.getUri() + "/Fully/QPath";

        +1 once addressed.

        Show
        vinayrpet Vinayakumar B added a comment - Looks almost Good. Just two nits. 1. testSnapshotCommandsWithURI() , add timeout ~60sec. 2. better to explain in comments why you using fresh config with different URI in below code. Configuration config = new HdfsConfiguration(); config.set( "fs.defaultFS" , "hdfs: //127.0.0.1:1024" ); String path = fs.getUri() + "/Fully/QPath" ; +1 once addressed.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the patch to address the above comments.. All testcases TestSnapshotCommands does not timeout..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch to address the above comments.. All testcases TestSnapshotCommands does not timeout..
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 4m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 2m 0s Maven dependency ordering for branch
        +1 mvninstall 13m 8s trunk passed
        +1 compile 14m 10s trunk passed
        +1 checkstyle 1m 54s trunk passed
        +1 mvnsite 2m 9s trunk passed
        +1 mvneclipse 0m 32s trunk passed
        +1 findbugs 3m 27s trunk passed
        +1 javadoc 1m 33s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 1m 37s the patch passed
        +1 compile 12m 19s the patch passed
        +1 javac 12m 19s the patch passed
        +1 checkstyle 1m 52s the patch passed
        +1 mvnsite 1m 56s the patch passed
        +1 mvneclipse 0m 38s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 27s the patch passed
        +1 javadoc 1m 33s the patch passed
        +1 unit 8m 37s hadoop-common in the patch passed.
        -1 unit 64m 30s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        141m 43s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestMaintenanceState



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11432
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855157/HDFS-11432-004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fe60f053ab1e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 989bd56
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18481/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18481/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18481/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 4m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 2m 0s Maven dependency ordering for branch +1 mvninstall 13m 8s trunk passed +1 compile 14m 10s trunk passed +1 checkstyle 1m 54s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 32s trunk passed +1 findbugs 3m 27s trunk passed +1 javadoc 1m 33s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 37s the patch passed +1 compile 12m 19s the patch passed +1 javac 12m 19s the patch passed +1 checkstyle 1m 52s the patch passed +1 mvnsite 1m 56s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 27s the patch passed +1 javadoc 1m 33s the patch passed +1 unit 8m 37s hadoop-common in the patch passed. -1 unit 64m 30s hadoop-hdfs in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 141m 43s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11432 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855157/HDFS-11432-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fe60f053ab1e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 989bd56 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18481/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18481/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18481/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Testfailures are unrelated..Vinayakumar B can you review the latest patch..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Testfailures are unrelated.. Vinayakumar B can you review the latest patch..?
        Hide
        vinayrpet Vinayakumar B added a comment -

        +1 for latest patch.
        Thanks Brahma Reddy Battula for the contribution.

        Show
        vinayrpet Vinayakumar B added a comment - +1 for latest patch. Thanks Brahma Reddy Battula for the contribution.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Committed to trunk,branch-2,branch-2.8..Vinayakumar B thanks a lot for your review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Committed to trunk,branch-2,branch-2.8.. Vinayakumar B thanks a lot for your review.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11325 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11325/)
        HDFS-11432. Federation : Support fully qualified path for (brahma: rev dcd03df9f9e0080d7e179060ffc8148336c31b3e)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CacheAdmin.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CryptoAdmin.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSnapshotCommands.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/SnapshotDiff.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11325 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11325/ ) HDFS-11432 . Federation : Support fully qualified path for (brahma: rev dcd03df9f9e0080d7e179060ffc8148336c31b3e) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CacheAdmin.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CryptoAdmin.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSnapshotCommands.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/SnapshotDiff.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
        Hide
        surendrasingh Surendra Singh Lilhore added a comment -

        Hi Brahma Reddy Battula,

        allowSnapshot: FileSystem viewfs://ClusterX/ is not an HDFS file system

        After this patch also still these commands will not work with "viewfs://" path. This patch is solving problem only for the fully qualified path with "hdfs" schema.
        I think we should change the jira description.

        Show
        surendrasingh Surendra Singh Lilhore added a comment - Hi Brahma Reddy Battula , allowSnapshot: FileSystem viewfs://ClusterX/ is not an HDFS file system After this patch also still these commands will not work with "viewfs://" path. This patch is solving problem only for the fully qualified path with "hdfs" schema. I think we should change the jira description.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          People

          • Assignee:
            brahmareddy Brahma Reddy Battula
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development