Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8241

Remove unused NameNode startup option -finalize

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Remove -finalize option from hdfs namenode command.

      Description

      Command : hdfs namenode -finalize

      15/04/24 22:26:23 INFO namenode.NameNode: createNameNode [-finalize]
      Use of the argument 'FINALIZE' is no longer supported. To finalize an upgrade, start the NN and then run `hdfs dfsadmin -finalizeUpgrade'

      1. HDFS-8241-003.patch
        7 kB
        Brahma Reddy Battula
      2. HDFS-8241-002.patch
        6 kB
        Brahma Reddy Battula
      3. HDFS-8241.patch
        4 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/193/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/193/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #183 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/183/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #183 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/183/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2123 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2123/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2123 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2123/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2141 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2141/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2141 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2141/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/194/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/194/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #925 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/925/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #925 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/925/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          aw Allen Wittenauer added a comment -

          because people wanted to be able to start new software even if they forgot to -finalizeUpgrade before shutting down the cluster.

          It's worth pointing out that starting the cluster took 4+ hours on 1000 nodes in 0.14 too. In other words, it was extremely expensive and a major cause of downtime.

          Show
          aw Allen Wittenauer added a comment - because people wanted to be able to start new software even if they forgot to -finalizeUpgrade before shutting down the cluster. It's worth pointing out that starting the cluster took 4+ hours on 1000 nodes in 0.14 too. In other words, it was extremely expensive and a major cause of downtime.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks Akira Ajisaka for committing and reviewing!!! I think, this can be checked-in branch-2 too, As HDFS-5138 is incompatibility change in 2.4.0 ( target version for HDFS-5138)..what do you think..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks Akira Ajisaka for committing and reviewing!!! I think, this can be checked-in branch-2 too, As HDFS-5138 is incompatibility change in 2.4.0 ( target version for HDFS-5138 )..what do you think..?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          TestTraceAdmin: Fails in trunk as well. I'll file a jira for this.

          Tracked by HDFS-8358.

          Show
          ajisakaa Akira Ajisaka added a comment - TestTraceAdmin: Fails in trunk as well. I'll file a jira for this. Tracked by HDFS-8358 .
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7791 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7791/)
          HDFS-8241. Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7791 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7791/ ) HDFS-8241 . Remove unused NameNode startup option -finalize. Contributed by Brahma Reddy Battula. (aajisaka: rev 1dd79ffaca4b0c2cb0ab817dff3697686f3367e3) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestHdfsServerConstants.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk. Thanks Brahma Reddy Battula for contribution, and thanks all who commented on this issue.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk. Thanks Brahma Reddy Battula for contribution, and thanks all who commented on this issue.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1. Committing this shortly.

          • TestHdfsConfigFields: tracked by HDFS-8362
          • TestTraceAdmin: Fails in trunk as well. I'll file a jira for this.
          Show
          ajisakaa Akira Ajisaka added a comment - +1. Committing this shortly. TestHdfsConfigFields: tracked by HDFS-8362 TestTraceAdmin: Fails in trunk as well. I'll file a jira for this.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 34s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 27s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 site 2m 58s Site still builds.
          +1 checkstyle 0m 38s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 11s Pre-build of native portion
          -1 hdfs tests 165m 41s Tests failed in hadoop-hdfs.
              212m 43s  



          Reason Tests
          Failed unit tests hadoop.hdfs.tools.TestHdfsConfigFields
            hadoop.tracing.TestTraceAdmin



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731883/HDFS-8241-003.patch
          Optional Tests javadoc javac unit findbugs checkstyle site
          git revision trunk / 3fa2efc
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10910/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10910/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10910/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 34s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 site 2m 58s Site still builds. +1 checkstyle 0m 38s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 165m 41s Tests failed in hadoop-hdfs.     212m 43s   Reason Tests Failed unit tests hadoop.hdfs.tools.TestHdfsConfigFields   hadoop.tracing.TestTraceAdmin Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731883/HDFS-8241-003.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 3fa2efc hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10910/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10910/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10910/console This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1 pending Jenkins. Thank you Brahma.

          Show
          ajisakaa Akira Ajisaka added a comment - +1 pending Jenkins. Thank you Brahma.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Akira Ajisaka Updated patch to fix above minor nit.. Kindly review

          Show
          brahmareddy Brahma Reddy Battula added a comment - Akira Ajisaka Updated patch to fix above minor nit.. Kindly review
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Brahma Reddy Battula for updating the patch. Minor nit:

          HDFSCommands.md
          Runs the namenode. More info about the upgrade, rollback and finalize is at [Upgrade Rollback](./HdfsUserGuide.html#Upgrade_and_Rollback).
          

          Would you remove finalize in the sentence?

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Brahma Reddy Battula for updating the patch. Minor nit: HDFSCommands.md Runs the namenode. More info about the upgrade, rollback and finalize is at [Upgrade Rollback](./HdfsUserGuide.html#Upgrade_and_Rollback). Would you remove finalize in the sentence?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Akira Ajisaka Updated patch..Kindly Review!! Thanks.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Akira Ajisaka Updated patch..Kindly Review!! Thanks.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Brahma Reddy Battula, I have additional two comments.

          1. Would you remove the option from enum HdfsServerConstants.StartupOption?
          2. Would you remove the option from document entirely? We have deprecated the option in the document of trunk and branch-2 by HDFS-8351, we can remove it in trunk.

          I'm +1 if these are addressed.

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Brahma Reddy Battula , I have additional two comments. Would you remove the option from enum HdfsServerConstants.StartupOption ? Would you remove the option from document entirely? We have deprecated the option in the document of trunk and branch-2 by HDFS-8351 , we can remove it in trunk. I'm +1 if these are addressed.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot Akira!!!

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot Akira!!!
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I'll wait until May 11 before committing since there're many watchers. Any comments are welcome.

          Show
          ajisakaa Akira Ajisaka added a comment - I'll wait until May 11 before committing since there're many watchers. Any comments are welcome.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I'm +1 for the patch. I'll file a separate jira for documentation.

          Show
          ajisakaa Akira Ajisaka added a comment - I'm +1 for the patch. I'll file a separate jira for documentation.
          Hide
          shv Konstantin Shvachko added a comment -

          Historically, initial implementation of upgrade HADOOP-702 did not have the finalize startup option, only the admin command. StartupOption.FINALIZE was requested later HADOOP-1604 because people wanted to be able to start new software even if they forgot to -finalizeUpgrade before shutting down the cluster.

          Show
          shv Konstantin Shvachko added a comment - Historically, initial implementation of upgrade HADOOP-702 did not have the finalize startup option, only the admin command. StartupOption.FINALIZE was requested later HADOOP-1604 because people wanted to be able to start new software even if they forgot to -finalizeUpgrade before shutting down the cluster.
          Hide
          sanjay.radia Sanjay Radia added a comment -

          Yes it was unfortunate that this incompatibility was missed.
          Q. do folks feel that a startup -finalize option to NN is a good interface in ADDITION to admin command to finalize?
          Clearly we needs the admin command to finalize since one does not want to restart the NN to finalize.

          Show
          sanjay.radia Sanjay Radia added a comment - Yes it was unfortunate that this incompatibility was missed. Q. do folks feel that a startup -finalize option to NN is a good interface in ADDITION to admin command to finalize? Clearly we needs the admin command to finalize since one does not want to restart the NN to finalize.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 27s There were no new javac warning messages.
          +1 javadoc 9m 33s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 7m 50s There were no new checkstyle issues.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 15s Pre-build of native portion
          +1 hdfs tests 170m 46s Tests passed in hadoop-hdfs.
              219m 4s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728302/HDFS-8241.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1a2459b
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10403/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10403/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10403/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 7m 50s There were no new checkstyle issues. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 15s Pre-build of native portion +1 hdfs tests 170m 46s Tests passed in hadoop-hdfs.     219m 4s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728302/HDFS-8241.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1a2459b hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10403/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10403/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10403/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks to all ..Attached the patch..kindly review..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks to all ..Attached the patch..kindly review..
          Hide
          shv Konstantin Shvachko added a comment -

          > I'm astounded as to how many people were watching that JIRA and still thought it was OK

          Well, not everybody, but Aaron seemed quite confident about changing startup options and their semantics.

          Show
          shv Konstantin Shvachko added a comment - > I'm astounded as to how many people were watching that JIRA and still thought it was OK Well, not everybody, but Aaron seemed quite confident about changing startup options and their semantics.
          Hide
          cwimmer Charles Wimmer added a comment -

          A fatal error would obviously be prefereable to a silent failure. It is also no longer obvious when a filesystem requires finalization.

          (I'll skip my rant about how HDFS-5138 was probably the cause for me losing blocks on an upgrade from 2.2 to 2.4.1)

          Show
          cwimmer Charles Wimmer added a comment - A fatal error would obviously be prefereable to a silent failure. It is also no longer obvious when a filesystem requires finalization. (I'll skip my rant about how HDFS-5138 was probably the cause for me losing blocks on an upgrade from 2.2 to 2.4.1)
          Hide
          aw Allen Wittenauer added a comment -

          HDFS-5138 was such a total screw-up. I'm astounded as to how many people were watching that JIRA and still thought it was OK to completely nuke operational procedures out of the water in the non-HA case.

          I've had discussions with Sanjay Radia to restore -finalize so that future generations of branch-2 users aren't caught by surprise. At this point, especially with branch-2 regularly tossing operations folks under the bus if it is worth the effort. It might be easier to go ahead and remove the -finalize option so that it's a fatal error rather than silently failing like it does now. Charles Wimmer, do you have any opinions on this?

          Show
          aw Allen Wittenauer added a comment - HDFS-5138 was such a total screw-up. I'm astounded as to how many people were watching that JIRA and still thought it was OK to completely nuke operational procedures out of the water in the non-HA case. I've had discussions with Sanjay Radia to restore -finalize so that future generations of branch-2 users aren't caught by surprise. At this point, especially with branch-2 regularly tossing operations folks under the bus if it is worth the effort. It might be easier to go ahead and remove the -finalize option so that it's a fatal error rather than silently failing like it does now. Charles Wimmer , do you have any opinions on this?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          HDFS-5138 removed -finalize option from 2.4.0 (incompatible change), so I'm thinking this issue can be applied to branch-2 if this issue is to remove the option from document. The option still exists in the document. (http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#namenode)

          Show
          ajisakaa Akira Ajisaka added a comment - HDFS-5138 removed -finalize option from 2.4.0 (incompatible change), so I'm thinking this issue can be applied to branch-2 if this issue is to remove the option from document. The option still exists in the document. ( http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#namenode )
          Hide
          andrew.wang Andrew Wang added a comment -

          I think we can only target this at trunk, since removing a flag is incompatible.

          Show
          andrew.wang Andrew Wang added a comment - I think we can only target this at trunk, since removing a flag is incompatible.

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development