Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      scenario:

      1) /ledgers/available zNode is not created in ZooKeeper.

      2) Apply NN formatting using : hadoop namenode -format

      Namenode formatting has dependency with the BookKeeper client and is throwing the following exception.

      12/05/10 16:41:57 ERROR namenode.NameNode: Exception in namenode join
      java.lang.IllegalArgumentException: Unable to construct journal, bookkeeper://10.18.40.155:2183;10.18.40.155:2182;10.18.40.155:2181/NN
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1226)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:242)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:223)
              at org.apache.hadoop.hdfs.server.namenode.FSImage.<init>(FSImage.java:134)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:718)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1091)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1191)
      Caused by: java.lang.reflect.InvocationTargetException
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
              at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1224)
              ... 6 more
      Caused by: java.io.IOException: Error initializing zk
              at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:191)
              ... 11 more
      Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
              at org.apache.bookkeeper.client.BookieWatcher.readBookiesBlocking(BookieWatcher.java:140)
              at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:177)
              at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:147)
              at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:189)
              ... 11 more
      
      1. BOOKKEEPER-255.patch
        5 kB
        Rakesh R
      2. HDFS-3408.1.patch
        6 kB
        Rakesh R
      3. HDFS-3408.2.patch
        9 kB
        Rakesh R
      4. HDFS-3408.3.patch
        9 kB
        Rakesh R
      5. HDFS-3408.4.patch
        10 kB
        Rakesh R

        Issue Links

          Activity

          Allen Wittenauer made changes -
          Fix Version/s 3.0.0 [ 12320356 ]
          Gavin made changes -
          Link This issue depends upon HDFS-3474 [ HDFS-3474 ]
          Gavin made changes -
          Link This issue depends on HDFS-3474 [ HDFS-3474 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Uma Maheswara Rao G made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.0.1-alpha [ 12321440 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Resolution Fixed [ 1 ]
          Rakesh R made changes -
          Attachment HDFS-3408.4.patch [ 12531809 ]
          Rakesh R made changes -
          Attachment HDFS-3408.3.patch [ 12530766 ]
          Rakesh R made changes -
          Attachment HDFS-3408.2.patch [ 12530566 ]
          Ivan Kelly made changes -
          Link This issue depends on HDFS-3474 [ HDFS-3474 ]
          Tsz Wo Nicholas Sze made changes -
          Assignee Rakesh R [ rakeshr ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.0-alpha [ 12320353 ]
          Rakesh R made changes -
          Fix Version/s 2.0.0 [ 12320353 ]
          Status Open [ 1 ] Patch Available [ 10002 ]
          Rakesh R made changes -
          Attachment HDFS-3408.1.patch [ 12527564 ]
          Flavio Junqueira made changes -
          Parent HDFS-3399 [ 12554717 ]
          Issue Type Task [ 3 ] Sub-task [ 7 ]
          Flavio Junqueira made changes -
          Project Bookkeeper [ 12311293 ] Hadoop HDFS [ 12310942 ]
          Key BOOKKEEPER-255 HDFS-3408
          Issue Type Bug [ 1 ] Task [ 3 ]
          Assignee Rakesh R [ rakeshr ]
          Component/s name-node [ 12312926 ]
          Component/s bookkeeper-client [ 12314393 ]
          Rakesh R made changes -
          Attachment BOOKKEEPER-255.patch [ 12526487 ]
          Rakesh R made changes -
          Field Original Value New Value
          Summary BKJM : Create BookKeeper parent nodes /ledgers/available znodes in ZK BKJM : Namenode format fails, if there is no BK root
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          Priority Major [ 3 ] Minor [ 4 ]
          Description BKJM should pre-create the BookKeeper parent znodes if not exists.

          bq. hadoop namenode -format

          Namenode formatting has dependency with the BookKeeper client and is throwing the folowing exception.

          {noformat}
          12/05/10 16:41:57 ERROR namenode.NameNode: Exception in namenode join
          java.lang.IllegalArgumentException: Unable to construct journal, bookkeeper://10.18.40.155:2183;10.18.40.155:2182;10.18.40.155:2181/NN
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1226)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:242)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:223)
                  at org.apache.hadoop.hdfs.server.namenode.FSImage.<init>(FSImage.java:134)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:718)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1091)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1191)
          Caused by: java.lang.reflect.InvocationTargetException
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
                  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
                  at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1224)
                  ... 6 more
          Caused by: java.io.IOException: Error initializing zk
                  at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:191)
                  ... 11 more
          Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
                  at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
                  at org.apache.bookkeeper.client.BookieWatcher.readBookiesBlocking(BookieWatcher.java:140)
                  at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:177)
                  at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:147)
                  at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:189)
                  ... 11 more
          {noformat}
          +scenario:+

          1) /ledgers/available zNode is not created in ZooKeeper.

          2) Apply NN formatting using : hadoop namenode -format

          Namenode formatting has dependency with the BookKeeper client and is throwing the following exception.

          {noformat}
          12/05/10 16:41:57 ERROR namenode.NameNode: Exception in namenode join
          java.lang.IllegalArgumentException: Unable to construct journal, bookkeeper://10.18.40.155:2183;10.18.40.155:2182;10.18.40.155:2181/NN
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1226)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initJournals(FSEditLog.java:242)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.initSharedJournalsForRead(FSEditLog.java:223)
                  at org.apache.hadoop.hdfs.server.namenode.FSImage.<init>(FSImage.java:134)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:718)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1091)
                  at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1191)
          Caused by: java.lang.reflect.InvocationTargetException
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
                  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
                  at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
                  at org.apache.hadoop.hdfs.server.namenode.FSEditLog.createJournal(FSEditLog.java:1224)
                  ... 6 more
          Caused by: java.io.IOException: Error initializing zk
                  at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:191)
                  ... 11 more
          Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
                  at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
                  at org.apache.bookkeeper.client.BookieWatcher.readBookiesBlocking(BookieWatcher.java:140)
                  at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:177)
                  at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:147)
                  at org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager.<init>(BookKeeperJournalManager.java:189)
                  ... 11 more
          {noformat}
          Rakesh R created issue -

            People

            • Assignee:
              Rakesh R
              Reporter:
              Rakesh R
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development