Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      scenario:

      1) /ledgers/available zNode is not created in ZooKeeper.

      2) Apply NN formatting using : hadoop namenode -format

      Namenode formatting has dependency with the BookKeeper client and is throwing the following exception.

      12/05/10 16:41:57 ERROR namenode.NameNode: Exception in namenode join
      java.lang.IllegalArgumentException: Unable to construct journal, bookkeeper://10.18.40.155:2183;10.18.40.155:2182;10.18.40.155:2181/NN
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1226)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:242)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:223)
              at org.apache.hadoop.hdfs.server.namenode.FSImage.<init>(FSImage.java:134)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:718)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1091)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1191)
      Caused by: java.lang.reflect.InvocationTargetException
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1224)
              ... 6 more
      Caused by: java.io.IOException: Error initializing zk
              at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:191)
              ... 11 more
      Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
              at org.apache.bookkeeper.client.BookieWatcher.readBookiesBlocking(BookieWatcher.java:140)
              at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:177)
              at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:147)
              at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:189)
              ... 11 more
      
      1. BOOKKEEPER-255.patch
        5 kB
        Rakesh R
      2. HDFS-3408.1.patch
        6 kB
        Rakesh R
      3. HDFS-3408.2.patch
        9 kB
        Rakesh R
      4. HDFS-3408.3.patch
        9 kB
        Rakesh R
      5. HDFS-3408.4.patch
        10 kB
        Rakesh R

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/)
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA) (Revision 1350733)

          Result = FAILURE
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/ ) Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) (Revision 1350733) Result = FAILURE umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/)
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA) (Revision 1350733)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/ ) Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) (Revision 1350733) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2380 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2380/)
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA) (Revision 1350733)

          Result = FAILURE
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2380 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2380/ ) Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) (Revision 1350733) Result = FAILURE umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2433 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2433/)
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA) (Revision 1350733)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2433 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2433/ ) Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) (Revision 1350733) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2361 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2361/)
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA) (Revision 1350733)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2361 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2361/ ) Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) (Revision 1350733) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          Uma Maheswara Rao G added a comment -

          Committed deatils in CHAGES.txt for trunk.

          URL: http://svn.apache.org/viewvc?rev=1350733&view=rev
          Log:
          Updated CHANGES.txt for HDFS-3408(BKJM : Namenode format fails, if there is no BK root) And HDFS-3389(Document the BKJM usage in Namenode HA)

          Modified:
          hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          Modified: hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt?rev=1350733&r1=1350732&r2=1350733&view=diff

          Show
          Uma Maheswara Rao G added a comment - Committed deatils in CHAGES.txt for trunk. URL: http://svn.apache.org/viewvc?rev=1350733&view=rev Log: Updated CHANGES.txt for HDFS-3408 (BKJM : Namenode format fails, if there is no BK root) And HDFS-3389 (Document the BKJM usage in Namenode HA) Modified: hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Modified: hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt?rev=1350733&r1=1350732&r2=1350733&view=diff
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1108 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1108/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456)

          Result = FAILURE
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java

          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1108 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1108/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456) Result = FAILURE umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1075 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1075/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java

          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1075 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1075/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Hide
          Uma Maheswara Rao G added a comment -

          Thanks a lot Eli for noticing, I will update that change.

          Show
          Uma Maheswara Rao G added a comment - Thanks a lot Eli for noticing, I will update that change.
          Hide
          Eli Collins added a comment -

          Hey Uma,k

          Looks like CHANGES.txt was updated on branch-2 but not on trunk. Mind fixing that so they're consistent?

          Thanks,
          Eli

          Show
          Eli Collins added a comment - Hey Uma,k Looks like CHANGES.txt was updated on branch-2 but not on trunk. Mind fixing that so they're consistent? Thanks, Eli
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2370 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2370/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459)

          Result = FAILURE
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2370 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2370/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459) Result = FAILURE umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2369 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2369/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456)

          Result = FAILURE
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2369 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2369/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456) Result = FAILURE umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2346 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2346/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java

          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2346 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2346/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2419 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2419/)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459)
          HDFS-3408. BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456)

          Result = SUCCESS
          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java

          umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2419 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2419/ ) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349459) HDFS-3408 . BKJM : Namenode format fails, if there is no BK root. Contributed by Rakesh R. (Revision 1349456) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349459 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1349456 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          Hide
          Uma Maheswara Rao G added a comment -

          I have just committed this to trunk and branch-2. Thanks a lot Rakesh for the patch. Thanks to Ivan for reviews!

          Show
          Uma Maheswara Rao G added a comment - I have just committed this to trunk and branch-2. Thanks a lot Rakesh for the patch. Thanks to Ivan for reviews!
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12531809/HDFS-3408.4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2645//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2645//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531809/HDFS-3408.4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2645//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2645//console This message is automatically generated.
          Hide
          Uma Maheswara Rao G added a comment -

          Thanks Rakesh for addressing all the comments. ZKUtils.createFullPathOptimistic should work for us.

          Latest patch looks good.

          +1

          Show
          Uma Maheswara Rao G added a comment - Thanks Rakesh for addressing all the comments. ZKUtils.createFullPathOptimistic should work for us. Latest patch looks good. +1
          Hide
          Rakesh R added a comment -

          Hi, there is a slight modification to the previous patch.

          In the latest patch HDF-3408.4, I 've found one bookkeeper utility - ZKUtils.createFullPathOptimistic and used the same in the latest patch. I just thought to reuse this utility rather than re-inventing. Please review.

          Show
          Rakesh R added a comment - Hi, there is a slight modification to the previous patch. In the latest patch HDF-3408.4, I 've found one bookkeeper utility - ZKUtils.createFullPathOptimistic and used the same in the latest patch. I just thought to reuse this utility rather than re-inventing. Please review.
          Hide
          Uma Maheswara Rao G added a comment -

          Patch looks great Rakesh.
          One small nit:
          Since this is local to the method, we could have used StringBuilder instead of string buffer? I missed it in my previous comment.

          StringBuffer zkAvailableFullPath = new StringBuffer(zkAvailablePath.length());
          

          +1 on addressing above.

          Show
          Uma Maheswara Rao G added a comment - Patch looks great Rakesh. One small nit: Since this is local to the method, we could have used StringBuilder instead of string buffer? I missed it in my previous comment. StringBuffer zkAvailableFullPath = new StringBuffer (zkAvailablePath.length()); +1 on addressing above.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12530766/HDFS-3408.3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2579//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2579//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530766/HDFS-3408.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2579//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2579//console This message is automatically generated.
          Hide
          Rakesh R added a comment -

          Thanks Uma for the review. I have uploaded new patch. Please review

          Show
          Rakesh R added a comment - Thanks Uma for the review. I have uploaded new patch. Please review
          Hide
          Uma Maheswara Rao G added a comment -
          String[] zkAvailablePath = conf.getStrings(BKJM_ZK_LEDGERS_AVAILABLE_PATH,
          +        BKJM_ZK_LEDGERS_AVAILABLE_PATH_DEFAULT);
          +
          +    String[] zkAvailFullPath = StringUtils.split(zkAvailablePath[0], '/');
          

          Why you are getting the zkAvailablePath using getString and using only zkAvailablePath[0]? Instead you can use get method right?

          Show
          Uma Maheswara Rao G added a comment - String [] zkAvailablePath = conf.getStrings(BKJM_ZK_LEDGERS_AVAILABLE_PATH, + BKJM_ZK_LEDGERS_AVAILABLE_PATH_DEFAULT); + + String [] zkAvailFullPath = StringUtils.split(zkAvailablePath[0], '/'); Why you are getting the zkAvailablePath using getString and using only zkAvailablePath [0] ? Instead you can use get method right?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12530566/HDFS-3408.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2567//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2567//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530566/HDFS-3408.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2567//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2567//console This message is automatically generated.
          Hide
          Rakesh R added a comment -

          Thanks Uma for the review. Updated latest patch by fixing the comments

          Show
          Rakesh R added a comment - Thanks Uma for the review. Updated latest patch by fixing the comments
          Hide
          Uma Maheswara Rao G added a comment -

          small comment:

           zkc.create(BOOKIE_ROOT_PATH, new byte[] {'0'}, Ids.OPEN_ACL_UNSAFE,
          +            CreateMode.PERSISTENT);
          

          we can handle NodeExistsException separately and ignore?

          Show
          Uma Maheswara Rao G added a comment - small comment: zkc.create(BOOKIE_ROOT_PATH, new byte [] {'0'}, Ids.OPEN_ACL_UNSAFE, + CreateMode.PERSISTENT); we can handle NodeExistsException separately and ignore?
          Hide
          Ivan Kelly added a comment -

          HDFS-3474 will remove the findbug issue. The current patch applies over it fine, though with a little offset change.

          Otherwise +1 on the fix from me.

          Show
          Ivan Kelly added a comment - HDFS-3474 will remove the findbug issue. The current patch applies over it fine, though with a little offset change. Otherwise +1 on the fix from me.
          Hide
          Rakesh R added a comment -

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          Findbugs issue is saying 'REC_CATCH_EXCEPTION', but this is not from this patch. This is an existing findbug issue present in the code base.

          The following catch block in BookKeeperJournalManager.java is causing the problem :
          "REC_CATCH_EXCEPTION:Exception is caught when Exception is not thrown"

              } catch (Exception e) {
                throw new IOException("Error initializing zk", e);
              }
          
          Show
          Rakesh R added a comment - -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. Findbugs issue is saying 'REC_CATCH_EXCEPTION', but this is not from this patch. This is an existing findbug issue present in the code base. The following catch block in BookKeeperJournalManager.java is causing the problem : "REC_CATCH_EXCEPTION:Exception is caught when Exception is not thrown" } catch (Exception e) { throw new IOException("Error initializing zk", e); }
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12527564/HDFS-3408.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//artifact/trunk/trunk/patchprocess/newPatchFindbugsWarningsbkjournal.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12527564/HDFS-3408.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//artifact/trunk/trunk/patchprocess/newPatchFindbugsWarningsbkjournal.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2447//console This message is automatically generated.
          Hide
          Rakesh R added a comment -

          Attached the latest patch. Please help me to review the changes and apply it to the code base. Thanks

          Show
          Rakesh R added a comment - Attached the latest patch. Please help me to review the changes and apply it to the code base. Thanks
          Hide
          Rakesh R added a comment -

          Thanks a lot.

          Oh! the attached patch doesn't have the license. I will attach the latest patch.

          Show
          Rakesh R added a comment - Thanks a lot. Oh! the attached patch doesn't have the license. I will attach the latest patch.
          Hide
          Uma Maheswara Rao G added a comment -

          We are not looking for Link option. Link would just link the issue, but the actual issue still will be in BK only. So, HDFS QA will not pick the patch.

          What i am looking is ("More actions" -> Move). Unfortunately I could not find that option in BK. Move will completely move the issue from BK project to HDFS.

          Show
          Uma Maheswara Rao G added a comment - We are not looking for Link option. Link would just link the issue, but the actual issue still will be in BK only. So, HDFS QA will not pick the patch. What i am looking is ("More actions" -> Move). Unfortunately I could not find that option in BK. Move will completely move the issue from BK project to HDFS.
          Hide
          Flavio Junqueira added a comment -

          I'm not sure what you guys are trying to do, and a bit of guidance might help me to fix it. I've checked that you can link this issue to an hdfs one ("More actions" -> Link). If this is not the sequence of steps you're looking for, let me know so that I can see how to enable it.

          Show
          Flavio Junqueira added a comment - I'm not sure what you guys are trying to do, and a bit of guidance might help me to fix it. I've checked that you can link this issue to an hdfs one ("More actions" -> Link). If this is not the sequence of steps you're looking for, let me know so that I can see how to enable it.
          Hide
          Uma Maheswara Rao G added a comment -

          Yes, even i don't see the option for movement of the issue from BK to other projects.
          I think Flavio/Ivan might have the access. We can move this to HDFS and convert it the subtask of HDFS-3399.

          Show
          Uma Maheswara Rao G added a comment - Yes, even i don't see the option for movement of the issue from BK to other projects. I think Flavio/Ivan might have the access. We can move this to HDFS and convert it the subtask of HDFS-3399 .
          Hide
          Rakesh R added a comment -

          Attached patch with test case.

          Show
          Rakesh R added a comment - Attached patch with test case.
          Hide
          Rakesh R added a comment -

          @Uma, I have raised in BK, becoz seen few BKJM issues moved from HDFS to BK

          Oh! there is no option available for moving this to HDFS. Please someone help me.

          Show
          Rakesh R added a comment - @Uma, I have raised in BK, becoz seen few BKJM issues moved from HDFS to BK Oh! there is no option available for moving this to HDFS. Please someone help me.
          Hide
          Uma Maheswara Rao G added a comment -

          Rakesh, This should be at HDFS side right?

          Show
          Uma Maheswara Rao G added a comment - Rakesh, This should be at HDFS side right?
          Hide
          Rakesh R added a comment -

          Here I feel the problem is, when we do NN formatting its going to intialize the configured BKJM journals and trying to create the BookKeeper client, this inturn reading the available Bookies.

          BK client looks for the available Bookie servers from ZK and is throwing exception.

          bk.getZkHandle().getChildren("/ledgers/available", this, callback, null);
          
          Show
          Rakesh R added a comment - Here I feel the problem is, when we do NN formatting its going to intialize the configured BKJM journals and trying to create the BookKeeper client, this inturn reading the available Bookies. BK client looks for the available Bookie servers from ZK and is throwing exception. bk.getZkHandle().getChildren("/ledgers/available", this, callback, null);

            People

            • Assignee:
              Rakesh R
              Reporter:
              Rakesh R
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development