Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This is a code refactoring JIRA in order to change Datanode to process all storage/data dirs in parallel; see also HDFS-8578.

      1. h9654_20160116.patch
        45 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - h9654_20160116.patch: the refactoring code from https://issues.apache.org/jira/secure/attachment/12777372/h8578_20151213.patch
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 7m 28s trunk passed
          +1 compile 0m 40s trunk passed with JDK v1.8.0_66
          +1 compile 0m 41s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 50s trunk passed
          +1 javadoc 1m 3s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 46s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 44s the patch passed
          +1 compile 0m 37s the patch passed with JDK v1.8.0_66
          +1 javac 0m 37s the patch passed
          +1 compile 0m 39s the patch passed with JDK v1.7.0_91
          +1 javac 0m 39s the patch passed
          -1 checkstyle 0m 16s Patch generated 2 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 166, now 150).
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 findbugs 2m 5s the patch passed
          +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 43s the patch passed with JDK v1.7.0_91
          -1 unit 64m 24s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 63m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          154m 0s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782727/h9654_20160116.patch
          JIRA Issue HDFS-9654
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 132e46068ee6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2a30386
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14140/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14140/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 7m 28s trunk passed +1 compile 0m 40s trunk passed with JDK v1.8.0_66 +1 compile 0m 41s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 1m 3s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 46s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 44s the patch passed +1 compile 0m 37s the patch passed with JDK v1.8.0_66 +1 javac 0m 37s the patch passed +1 compile 0m 39s the patch passed with JDK v1.7.0_91 +1 javac 0m 39s the patch passed -1 checkstyle 0m 16s Patch generated 2 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 166, now 150). +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 5s the patch passed +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 43s the patch passed with JDK v1.7.0_91 -1 unit 64m 24s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 63m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 154m 0s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782727/h9654_20160116.patch JIRA Issue HDFS-9654 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 132e46068ee6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a30386 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14140/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14140/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14140/console This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks for the refactor Tsz Wo Nicholas Sze.

          Changes looks quite good.

          Nit:

          +    final String name = "block pool " + blockpoolID + " at " + bpSd.getRoot();
          +    doUgrade(name, bpSd, nsInfo, bpPrevDir, bpTmpDir, bpCurDir, oldLV, conf);
          +  }
          +
          +  private void doUgrade(String name, final StorageDirectory bpSd,
          +      NamespaceInfo nsInfo, final File bpPrevDir, final File bpTmpDir,
          +      final File bpCurDir, final int oldLV, Configuration conf)
          +          throws IOException {
          

          In the above code change, Nit: doUgrade.
          Even, is this extraction to a method required, since its not used anywhere else.? In both BPSS and DataStorage

          Show
          vinayrpet Vinayakumar B added a comment - Thanks for the refactor Tsz Wo Nicholas Sze . Changes looks quite good. Nit: + final String name = "block pool " + blockpoolID + " at " + bpSd.getRoot(); + doUgrade(name, bpSd, nsInfo, bpPrevDir, bpTmpDir, bpCurDir, oldLV, conf); + } + + private void doUgrade( String name, final StorageDirectory bpSd, + NamespaceInfo nsInfo, final File bpPrevDir, final File bpTmpDir, + final File bpCurDir, final int oldLV, Configuration conf) + throws IOException { In the above code change, Nit: doUgrade . Even, is this extraction to a method required, since its not used anywhere else.? In both BPSS and DataStorage
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks Vinayakumar B for reviewing the patch.

          > Even, is this extraction to a method required, since its not used anywhere else.? In both BPSS and DataStorage

          It is not very useful in this patch but it will become useful in HDFS-8578 since doUpgrade will be invoked inside and outside Callable.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks Vinayakumar B for reviewing the patch. > Even, is this extraction to a method required, since its not used anywhere else.? In both BPSS and DataStorage It is not very useful in this patch but it will become useful in HDFS-8578 since doUpgrade will be invoked inside and outside Callable.
          Hide
          ctrezzo Chris Trezzo added a comment -

          Thanks Tsz Wo Nicholas Sze for the updated patch. Overall it is looking good! A few nits:

          1. Nit: Params in javadoc do not match the params for the method (i.e. missing config). These are some examples, there might be some others as well:

          BlockPoolSliceStorage

          147      private StorageDirectory loadStorageDirectory(NamespaceInfo nsInfo,
          148          File dataDir, StartupOption startOpt, Configuration conf)
          149              throws IOException {
          
          209      List<StorageDirectory> loadBpStorageDirectories(NamespaceInfo nsInfo,
          210          Collection<File> dataDirs, StartupOption startOpt,
          211          Configuration conf) throws IOException {
          
          244      List<StorageDirectory> recoverTransitionRead(NamespaceInfo nsInfo,
          245          Collection<File> dataDirs, StartupOption startOpt, Configuration conf)
          246              throws IOException {
          
          355      private boolean doTransition(StorageDirectory sd, NamespaceInfo nsInfo,
          356          StartupOption startOpt, Configuration conf) throws IOException {
          
          427      private void doUpgrade(final StorageDirectory bpSd,
          428          final NamespaceInfo nsInfo, final Configuration conf) throws IOException {
          
          658      private static void linkAllBlocks(File fromDir, File toDir,
          659          int diskLayoutVersion, Configuration conf) throws IOException {
          

          DataStorage

          661      private boolean doTransition(StorageDirectory sd, NamespaceInfo nsInfo,
          662          StartupOption startOpt, Configuration conf) throws IOException {
          
          734      void doUpgrade(final StorageDirectory sd, final NamespaceInfo nsInfo,
          735          final Configuration conf) throws IOException {
          

          2. Nit: Maybe make doUgrade method name a little more descriptive? How about hardLinkAndRename?:

          469      private void doUgrade(String name, final StorageDirectory bpSd,
          470          NamespaceInfo nsInfo, final File bpPrevDir, final File bpTmpDir,
          471          final File bpCurDir, final int oldLV, Configuration conf)
          472              throws IOException {
          

          3. Nit: the two checkstyle warnings for number of parameters in doUgrade. If we want to follow the checkstyle to the letter, then we could move this method into the callable annonymous inner class. If you want we can fix this in the next patch where we actually parallelize the method, whatever you think is best for this patch.

          4. Whitespace issue

          5. TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure test failure. Is this related?

          Thanks!

          Show
          ctrezzo Chris Trezzo added a comment - Thanks Tsz Wo Nicholas Sze for the updated patch. Overall it is looking good! A few nits: 1. Nit: Params in javadoc do not match the params for the method (i.e. missing config). These are some examples, there might be some others as well: BlockPoolSliceStorage 147 private StorageDirectory loadStorageDirectory(NamespaceInfo nsInfo, 148 File dataDir, StartupOption startOpt, Configuration conf) 149 throws IOException { 209 List<StorageDirectory> loadBpStorageDirectories(NamespaceInfo nsInfo, 210 Collection<File> dataDirs, StartupOption startOpt, 211 Configuration conf) throws IOException { 244 List<StorageDirectory> recoverTransitionRead(NamespaceInfo nsInfo, 245 Collection<File> dataDirs, StartupOption startOpt, Configuration conf) 246 throws IOException { 355 private boolean doTransition(StorageDirectory sd, NamespaceInfo nsInfo, 356 StartupOption startOpt, Configuration conf) throws IOException { 427 private void doUpgrade(final StorageDirectory bpSd, 428 final NamespaceInfo nsInfo, final Configuration conf) throws IOException { 658 private static void linkAllBlocks(File fromDir, File toDir, 659 int diskLayoutVersion, Configuration conf) throws IOException { DataStorage 661 private boolean doTransition(StorageDirectory sd, NamespaceInfo nsInfo, 662 StartupOption startOpt, Configuration conf) throws IOException { 734 void doUpgrade(final StorageDirectory sd, final NamespaceInfo nsInfo, 735 final Configuration conf) throws IOException { 2. Nit: Maybe make doUgrade method name a little more descriptive? How about hardLinkAndRename?: 469 private void doUgrade(String name, final StorageDirectory bpSd, 470 NamespaceInfo nsInfo, final File bpPrevDir, final File bpTmpDir, 471 final File bpCurDir, final int oldLV, Configuration conf) 472 throws IOException { 3. Nit: the two checkstyle warnings for number of parameters in doUgrade. If we want to follow the checkstyle to the letter, then we could move this method into the callable annonymous inner class. If you want we can fix this in the next patch where we actually parallelize the method, whatever you think is best for this patch. 4. Whitespace issue 5. TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure test failure. Is this related? Thanks!
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > 1. Nit: Params in javadoc do not match the params for the method (i.e. missing config). ...

          These are not public APIs so that we don't require perfect javadoc. We usually add javadoc/comment when the code is tricky.

          > 3. Nit: the two checkstyle warnings for number of parameters in doUgrade. ...
          > 4. Whitespace issue

          We could safely ignore the checkstyle warnings when they are very minor or unreasonable.

          > 5. TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure test failure. Is this related?

          It does not seem related. It also failed on some other build such as

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > 1. Nit: Params in javadoc do not match the params for the method (i.e. missing config). ... These are not public APIs so that we don't require perfect javadoc. We usually add javadoc/comment when the code is tricky. > 3. Nit: the two checkstyle warnings for number of parameters in doUgrade. ... > 4. Whitespace issue We could safely ignore the checkstyle warnings when they are very minor or unreasonable. > 5. TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure test failure. Is this related? It does not seem related. It also failed on some other build such as https://builds.apache.org/job/PreCommit-HDFS-Build/14245/testReport/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailure/testUnderReplicationAfterVolFailure/
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > 2. Nit: Maybe make doUgrade method name a little more descriptive? How about hardLinkAndRename?:

          I was going to rename the doUgrade method. However, it does do more than hardLink and rename. It additionally does set cTime and writeProperties. It won't looks good if we rename it to hardLinkAndSetCTimeAndWritePropertiesAndRename. We simply cannot encode everything in the method name.

          I also was going to fix the trailing whitespaces using git, although I always believe that we should focus on the non-whitespace characters and ignore the whitespaces, especially the trailing whitespaces. Since there are no other changes, I won't post a new patch. We could commit the patch by "--whitespace=fix" command.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > 2. Nit: Maybe make doUgrade method name a little more descriptive? How about hardLinkAndRename?: I was going to rename the doUgrade method. However, it does do more than hardLink and rename. It additionally does set cTime and writeProperties. It won't looks good if we rename it to hardLinkAndSetCTimeAndWritePropertiesAndRename. We simply cannot encode everything in the method name. I also was going to fix the trailing whitespaces using git, although I always believe that we should focus on the non-whitespace characters and ignore the whitespaces, especially the trailing whitespaces. Since there are no other changes, I won't post a new patch. We could commit the patch by "--whitespace=fix" command.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks Chris Trezzo for reviewing the patch!

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks Chris Trezzo for reviewing the patch!
          Hide
          ctrezzo Chris Trezzo added a comment -

          Sounds good! +1 on the patch. Thanks Tsz Wo Nicholas Sze.

          Show
          ctrezzo Chris Trezzo added a comment - Sounds good! +1 on the patch. Thanks Tsz Wo Nicholas Sze .
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9198 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9198/)
          HDFS-9654. Code refactoring for HDFS-8578. (szetszwo: rev 662e17b46a0f41ade6a304e12925b70b5d09fc2f)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StorageLocation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9198 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9198/ ) HDFS-9654 . Code refactoring for HDFS-8578 . (szetszwo: rev 662e17b46a0f41ade6a304e12925b70b5d09fc2f) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StorageLocation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I have committed this.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I have committed this.
          Hide
          kihwal Kihwal Lee added a comment -

          This patch will fail unit tests when we increment the datanode layout version next time.

          TestDatanodeStartupFixesLegacyStorageIDs#testUpgradeFrom22via26FixesStorageIDs
          TestDatanodeStartupFixesLegacyStorageIDs#testUpgradeFrom22FixesStorageIDs
          

          This is because createStorageID() is no longer called when it goes through the layout upgrade path. We saw this failure because we have been running with the new datanode layout change.

          Show
          kihwal Kihwal Lee added a comment - This patch will fail unit tests when we increment the datanode layout version next time. TestDatanodeStartupFixesLegacyStorageIDs#testUpgradeFrom22via26FixesStorageIDs TestDatanodeStartupFixesLegacyStorageIDs#testUpgradeFrom22FixesStorageIDs This is because createStorageID() is no longer called when it goes through the layout upgrade path. We saw this failure because we have been running with the new datanode layout change.
          Hide
          kihwal Kihwal Lee added a comment -

          Filed HDFS-9730.

          Show
          kihwal Kihwal Lee added a comment - Filed HDFS-9730 .
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development