Details

    • Type: Sub-task
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: HDFS-10467
    • Component/s: fs
    • Labels:
      None

      Description

      Adding configuration options in tabular format.

      1. HDFS-12381-HDFS-10467.000.patch
        5 kB
        Íñigo Goiri
      2. HDFS-12381-HDFS-10467.001.patch
        8 kB
        Íñigo Goiri
      3. HDFS-12381-HDFS-10467.002.patch
        12 kB
        Íñigo Goiri
      4. HDFS-12381-HDFS-10467.003.patch
        13 kB
        Íñigo Goiri

        Issue Links

          Activity

          Hide
          elgoiri Íñigo Goiri added a comment -

          As Brahma Reddy Battula suggested, I will add a tabular with all the configuration options.

          Show
          elgoiri Íñigo Goiri added a comment - As Brahma Reddy Battula suggested, I will add a tabular with all the configuration options.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 28s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
                HDFS-10467 Compile Tests
          -1 mvninstall 15m 53s root in HDFS-10467 failed.
          +1 mvnsite 1m 8s HDFS-10467 passed
                Patch Compile Tests
          +1 mvnsite 0m 59s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
                Other Tests
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          19m 2s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12381
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884855/HDFS-12381-HDFS-10467.000.patch
          Optional Tests asflicense mvnsite
          uname Linux d770de1e4e4a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision HDFS-10467 / fc2c254
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/20959/artifact/patchprocess/branch-mvninstall-root.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20959/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags.       HDFS-10467 Compile Tests -1 mvninstall 15m 53s root in HDFS-10467 failed. +1 mvnsite 1m 8s HDFS-10467 passed       Patch Compile Tests +1 mvnsite 0m 59s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues.       Other Tests +1 asflicense 0m 15s The patch does not generate ASF License warnings. 19m 2s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12381 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884855/HDFS-12381-HDFS-10467.000.patch Optional Tests asflicense mvnsite uname Linux d770de1e4e4a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HDFS-10467 / fc2c254 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/20959/artifact/patchprocess/branch-mvninstall-root.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20959/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Given the comment from Manoj Govindassamy about recommendations/best-practices about setting up the mount table, I think I can extend this JIRA to cover this a little more.

          Show
          elgoiri Íñigo Goiri added a comment - Given the comment from Manoj Govindassamy about recommendations/best-practices about setting up the mount table, I think I can extend this JIRA to cover this a little more.
          Hide
          manojg Manoj Govindassamy added a comment -

          Thanks Inigo. Previously attached patch has good details. Additional config related items will be very useful.

          Show
          manojg Manoj Govindassamy added a comment - Thanks Inigo . Previously attached patch has good details. Additional config related items will be very useful.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Íñigo Goiri thanks for reporting and working on this.

          Minor nits:

          1) Please correct the typo's

          182 | Propery | Description|

          2) How about chage like "If true ,the Router should monitor the namenode in the local machine" which will be sync with existing conf .If you agree, please change hdfs-default.xml and remaining configs also.

          | dfs.federation.router.monitor.localnamenode.enable | If the Router should monitor the namenode in the local machine. |

          3) Are you going update with this some more details..?

          recommendations/best-practices about setting up the mount table, I think I can extend this JIRA to cover this a little more.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Íñigo Goiri thanks for reporting and working on this. Minor nits: 1) Please correct the typo's 182 | Propery | Description| 2) How about chage like "If true ,the Router should monitor the namenode in the local machine" which will be sync with existing conf .If you agree, please change hdfs-default.xml and remaining configs also. | dfs.federation.router.monitor.localnamenode.enable | If the Router should monitor the namenode in the local machine. | 3) Are you going update with this some more details..? recommendations/best-practices about setting up the mount table, I think I can extend this JIRA to cover this a little more.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Thanks Brahma Reddy Battula for the comments.
          I pushed a new patch with most of them.
          Not super proud of the best-practices part so I'm open to suggestions.

          Show
          elgoiri Íñigo Goiri added a comment - Thanks Brahma Reddy Battula for the comments. I pushed a new patch with most of them. Not super proud of the best-practices part so I'm open to suggestions.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                HDFS-10467 Compile Tests
          -1 mvninstall 4m 22s root in HDFS-10467 failed.
          -1 compile 0m 29s hadoop-hdfs in HDFS-10467 failed.
          -1 mvnsite 0m 29s hadoop-hdfs in HDFS-10467 failed.
          +1 javadoc 0m 46s HDFS-10467 passed
                Patch Compile Tests
          -1 mvninstall 0m 28s hadoop-hdfs in the patch failed.
          -1 compile 0m 28s hadoop-hdfs in the patch failed.
          -1 javac 0m 28s hadoop-hdfs in the patch failed.
          -1 mvnsite 0m 29s hadoop-hdfs in the patch failed.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 47s the patch passed
                Other Tests
          -1 unit 0m 30s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          10m 35s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HDFS-12381
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886681/HDFS-12381-HDFS-10467.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux ad85c1e10760 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision HDFS-10467 / 1a4ced3
          Default Java 1.8.0_144
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-mvninstall-root.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21098/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21098/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       HDFS-10467 Compile Tests -1 mvninstall 4m 22s root in HDFS-10467 failed. -1 compile 0m 29s hadoop-hdfs in HDFS-10467 failed. -1 mvnsite 0m 29s hadoop-hdfs in HDFS-10467 failed. +1 javadoc 0m 46s HDFS-10467 passed       Patch Compile Tests -1 mvninstall 0m 28s hadoop-hdfs in the patch failed. -1 compile 0m 28s hadoop-hdfs in the patch failed. -1 javac 0m 28s hadoop-hdfs in the patch failed. -1 mvnsite 0m 29s hadoop-hdfs in the patch failed. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 47s the patch passed       Other Tests -1 unit 0m 30s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 10m 35s Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12381 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886681/HDFS-12381-HDFS-10467.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux ad85c1e10760 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HDFS-10467 / 1a4ced3 Default Java 1.8.0_144 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21098/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21098/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21098/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          elgoiri Íñigo Goiri added a comment -

          The errors are caused by modifying hdfs-default.xml.
          I'll ignore the output from jenkins.

          Show
          elgoiri Íñigo Goiri added a comment - The errors are caused by modifying hdfs-default.xml . I'll ignore the output from jenkins.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          There are compilations errors , can you look once? I am on mobile.Errors might not related to this jira.

          Show
          brahmareddy Brahma Reddy Battula added a comment - There are compilations errors , can you look once? I am on mobile.Errors might not related to this jira.
          Hide
          elgoiri Íñigo Goiri added a comment -

          The error is:

          RouterRpcServer.java:[138,7] error: RouterRpcServer is not abstract and does not override abstract method getECBlockGroupStats() in ClientProtocol
          

          Apparently there is yet one more method addition to ClientProtocol related to EC.
          This is weird because HDFS-10467 was working and I haven't rebased.
          I'll post another rebase JIRA for HDFS-10467.

          Show
          elgoiri Íñigo Goiri added a comment - The error is: RouterRpcServer.java:[138,7] error: RouterRpcServer is not abstract and does not override abstract method getECBlockGroupStats() in ClientProtocol Apparently there is yet one more method addition to ClientProtocol related to EC. This is weird because HDFS-10467 was working and I haven't rebased. I'll post another rebase JIRA for HDFS-10467 .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Apparently there is yet one more method addition to ClientProtocol related to EC.

          Yes, it's after HDFS-12381. we need to rebase.

          Compilation Errors for reference

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project h
          adoop-hdfs: Compilation failure: Compilation failure:
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[69,39] cannot find symbol
          [ERROR]   symbol:   class BlocksStats
          [ERROR]   location: package org.apache.hadoop.hdfs.protocol
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[79,39] cannot find symbol
          [ERROR]   symbol:   class ECBlockGroupsStats
          [ERROR]   location: package org.apache.hadoop.hdfs.protocol
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[1882,10] cannot find symbol
          [ERROR]   symbol:   class ECBlockGroupsStats
          [ERROR]   location: class org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[1894,10] cannot find symbol
          [ERROR]   symbol:   class BlocksStats
          [ERROR]   location: class org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[138,8] org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer is not abstract and doe
          s not override abstract method getECBlockGroupStats() in org.apache.hadoop.hdfs.protocol.ClientProtocol
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[1881,3] method does not override or implement a method from a supertype
          [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou
          ter/RouterRpcServer.java:[1893,3] method does not override or implement a method from a supertype
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - Apparently there is yet one more method addition to ClientProtocol related to EC. Yes, it's after HDFS-12381 . we need to rebase. Compilation Errors for reference [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project h adoop-hdfs: Compilation failure: Compilation failure: [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[69,39] cannot find symbol [ERROR] symbol: class BlocksStats [ERROR] location: package org.apache.hadoop.hdfs.protocol [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[79,39] cannot find symbol [ERROR] symbol: class ECBlockGroupsStats [ERROR] location: package org.apache.hadoop.hdfs.protocol [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[1882,10] cannot find symbol [ERROR] symbol: class ECBlockGroupsStats [ERROR] location: class org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[1894,10] cannot find symbol [ERROR] symbol: class BlocksStats [ERROR] location: class org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[138,8] org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer is not abstract and doe s not override abstract method getECBlockGroupStats() in org.apache.hadoop.hdfs.protocol.ClientProtocol [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[1881,3] method does not override or implement a method from a supertype [ERROR] /D:/HDFS-10467/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/rou ter/RouterRpcServer.java:[1893,3] method does not override or implement a method from a supertype
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                HDFS-10467 Compile Tests
          +1 mvninstall 21m 4s HDFS-10467 passed
          +1 compile 1m 10s HDFS-10467 passed
          +1 mvnsite 1m 18s HDFS-10467 passed
          +1 javadoc 1m 0s HDFS-10467 passed
                Patch Compile Tests
          +1 mvninstall 1m 13s the patch passed
          +1 compile 0m 55s the patch passed
          +1 javac 0m 55s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 43s the patch passed
                Other Tests
          -1 unit 98m 58s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          129m 19s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestReconstructStripedFile
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.TestWriteRead
            hadoop.hdfs.TestSafeModeWithStripedFile
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.TestSeekBug
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.namenode.TestReencryptionWithKMS
            hadoop.hdfs.server.federation.router.TestNamenodeHeartbeat
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HDFS-12381
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886990/HDFS-12381-HDFS-10467.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 71f39e4f7663 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision HDFS-10467 / 679e31a
          Default Java 1.8.0_144
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21130/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21130/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21130/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       HDFS-10467 Compile Tests +1 mvninstall 21m 4s HDFS-10467 passed +1 compile 1m 10s HDFS-10467 passed +1 mvnsite 1m 18s HDFS-10467 passed +1 javadoc 1m 0s HDFS-10467 passed       Patch Compile Tests +1 mvninstall 1m 13s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 43s the patch passed       Other Tests -1 unit 98m 58s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 129m 19s Reason Tests Failed junit tests hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestWriteRead   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestSeekBug   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.TestReencryptionWithKMS   hadoop.hdfs.server.federation.router.TestNamenodeHeartbeat Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12381 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886990/HDFS-12381-HDFS-10467.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 71f39e4f7663 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HDFS-10467 / 679e31a Default Java 1.8.0_144 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21130/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21130/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21130/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 lgtm. Brahma Reddy Battula, Manoj Govindassamy: any other feedback?

          Show
          chris.douglas Chris Douglas added a comment - +1 lgtm. Brahma Reddy Battula , Manoj Govindassamy : any other feedback?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Íñigo Goiri thanks for updating the patch.
          I am thinking following also,sorry to trouble you.

          1) can we add stop command for router..? may be start command also update like " hdfs --daemon start router"..?
          2) how about adding default value to each property in the table(so totally three colums)..? and change description remaining boolean as i mentioned in earlier comment..?
          3) Fix following two typos also,like below..?

          Advanced functions like snapshotting, encryption

          Advanced functions like snapshot, encryption

          Adminstrators can query information

          Administrators can query information

          Show
          brahmareddy Brahma Reddy Battula added a comment - Íñigo Goiri thanks for updating the patch. I am thinking following also,sorry to trouble you. 1) can we add stop command for router..? may be start command also update like " hdfs --daemon start router"..? 2) how about adding default value to each property in the table(so totally three colums)..? and change description remaining boolean as i mentioned in earlier comment..? 3) Fix following two typos also,like below..? Advanced functions like snapshotting, encryption Advanced functions like snapshot, encryption Adminstrators can query information Administrators can query information
          Hide
          elgoiri Íñigo Goiri added a comment - - edited

          #2 and #3 sound good.

          For #1, you are talking about adding this in the documentation?
          If so, that's OK with me.
          In the deployment section or somewhere else?

          Feel free to ask for any other documentation addition.
          I've been heads down with this for a while and everything seems obvious to me and I'm assuming a lot of things.
          The easier it is to understand, the more chances for this to be adopted.

          Show
          elgoiri Íñigo Goiri added a comment - - edited #2 and #3 sound good. For #1, you are talking about adding this in the documentation? If so, that's OK with me. In the deployment section or somewhere else? Feel free to ask for any other documentation addition. I've been heads down with this for a while and everything seems obvious to me and I'm assuming a lot of things. The easier it is to understand, the more chances for this to be adopted.
          Hide
          manojg Manoj Govindassamy added a comment -

          Overall LGTM. +1.

          For example, if we want a federated address called `/data/wl1`, it is recommended to have that same name in the destination namespace.

          • Federated address is not consistent with the wordings 'federated folder' mentioned in the previous line. Also for destination namespace. Either we can call it address by including the schema, authority, etc., or may be call it just federated address? Your thoughts?
          • It would be really helpful if we can touch upon the non-existing mount points as well.
          Show
          manojg Manoj Govindassamy added a comment - Overall LGTM. +1. For example, if we want a federated address called `/data/wl1`, it is recommended to have that same name in the destination namespace. Federated address is not consistent with the wordings 'federated folder' mentioned in the previous line. Also for destination namespace. Either we can call it address by including the schema, authority, etc., or may be call it just federated address? Your thoughts? It would be really helpful if we can touch upon the non-existing mount points as well.
          Hide
          elgoiri Íñigo Goiri added a comment - - edited

          Thanks for the comments Brahma Reddy Battula and Manoj Govindassamy.
          From your comments, I realized that I wasn't talking much about the client, I added a full section on that.
          At the same time, I think I covered most of your comments.
          Let me know if there is anything else you think should be documented.

          Chris Douglas I added a note on security being work in progress.
          Does that seems reasonable as a placeholder until HDFS-12284 is complete?

          Show
          elgoiri Íñigo Goiri added a comment - - edited Thanks for the comments Brahma Reddy Battula and Manoj Govindassamy . From your comments, I realized that I wasn't talking much about the client, I added a full section on that. At the same time, I think I covered most of your comments. Let me know if there is anything else you think should be documented. Chris Douglas I added a note on security being work in progress. Does that seems reasonable as a placeholder until HDFS-12284 is complete?
          Hide
          chris.douglas Chris Douglas added a comment -

          I added a note on security being work in progress.

          For user docs, might as well spell it out instead of linking into JIRA. I'd also move the notice closer to the top of the page, rather than the bottom. Something like:

          Secure authentication and authorization are not supported yet, so the Router will not proxy to Hadoop clusters with security enabled.
          
          Show
          chris.douglas Chris Douglas added a comment - I added a note on security being work in progress. For user docs, might as well spell it out instead of linking into JIRA. I'd also move the notice closer to the top of the page, rather than the bottom. Something like: Secure authentication and authorization are not supported yet, so the Router will not proxy to Hadoop clusters with security enabled.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 2m 56s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                HDFS-10467 Compile Tests
          +1 mvninstall 14m 13s HDFS-10467 passed
          +1 compile 0m 52s HDFS-10467 passed
          +1 mvnsite 0m 56s HDFS-10467 passed
          +1 javadoc 0m 53s HDFS-10467 passed
                Patch Compile Tests
          +1 mvninstall 0m 57s the patch passed
          +1 compile 0m 48s the patch passed
          +1 javac 0m 48s the patch passed
          +1 mvnsite 0m 55s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 43s the patch passed
                Other Tests
          -1 unit 96m 10s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          120m 54s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestLeaseRecoveryStriped
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HDFS-12381
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887455/HDFS-12381-HDFS-10467.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux a02b1f7279a6 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision HDFS-10467 / 2d490d3
          Default Java 1.8.0_144
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21179/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21179/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21179/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 2m 56s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       HDFS-10467 Compile Tests +1 mvninstall 14m 13s HDFS-10467 passed +1 compile 0m 52s HDFS-10467 passed +1 mvnsite 0m 56s HDFS-10467 passed +1 javadoc 0m 53s HDFS-10467 passed       Patch Compile Tests +1 mvninstall 0m 57s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed +1 mvnsite 0m 55s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 43s the patch passed       Other Tests -1 unit 96m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 120m 54s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestLeaseRecoveryStriped Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12381 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887455/HDFS-12381-HDFS-10467.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux a02b1f7279a6 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HDFS-10467 / 2d490d3 Default Java 1.8.0_144 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21179/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21179/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21179/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 lgtm

          Show
          chris.douglas Chris Douglas added a comment - +1 lgtm
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                HDFS-10467 Compile Tests
          +1 mvninstall 18m 34s HDFS-10467 passed
          +1 compile 0m 50s HDFS-10467 passed
          +1 mvnsite 0m 57s HDFS-10467 passed
          +1 javadoc 0m 46s HDFS-10467 passed
                Patch Compile Tests
          +1 mvninstall 0m 51s the patch passed
          +1 compile 0m 48s the patch passed
          +1 javac 0m 48s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 44s the patch passed
                Other Tests
          -1 unit 83m 21s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          109m 31s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HDFS-12381
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887532/HDFS-12381-HDFS-10467.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux c54443bb85a4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision HDFS-10467 / 2d490d3
          Default Java 1.8.0_144
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21186/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21186/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21186/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       HDFS-10467 Compile Tests +1 mvninstall 18m 34s HDFS-10467 passed +1 compile 0m 50s HDFS-10467 passed +1 mvnsite 0m 57s HDFS-10467 passed +1 javadoc 0m 46s HDFS-10467 passed       Patch Compile Tests +1 mvninstall 0m 51s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 44s the patch passed       Other Tests -1 unit 83m 21s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 109m 31s Reason Tests Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.TestClientProtocolForPipelineRecovery Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12381 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887532/HDFS-12381-HDFS-10467.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux c54443bb85a4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HDFS-10467 / 2d490d3 Default Java 1.8.0_144 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21186/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21186/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21186/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Test Failures are unrelated,me too +1 on latest patch.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Test Failures are unrelated,me too +1 on latest patch.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Manoj Govindassamy, do you have any other comments?

          Show
          elgoiri Íñigo Goiri added a comment - Manoj Govindassamy , do you have any other comments?

            People

            • Assignee:
              elgoiri Íñigo Goiri
              Reporter:
              elgoiri Íñigo Goiri
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development