Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3399 BookKeeper option support for NN HA
  3. HDFS-3423

BKJM: NN startup is failing, when tries to recoverUnfinalizedSegments() a bad inProgress_ ZNodes

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None

      Description

      Say, the InProgress_000X node is corrupted due to not writing the data(version, ledgerId, firstTxId) to this inProgress_000X znode. Namenode startup has the logic to recover all the unfinalized segments, here will try to read the segment and getting shutdown.

      EditLogLedgerMetadata.java:
      
      static EditLogLedgerMetadata read(ZooKeeper zkc, String path)
            throws IOException, KeeperException.NoNodeException  {
            byte[] data = zkc.getData(path, false, null);
            String[] parts = new String(data).split(";");
            if (parts.length == 3)
               ....reading inprogress metadata
            else if (parts.length == 4)
               ....reading inprogress metadata
            else
              throw new IOException("Invalid ledger entry, "
                                    + new String(data));
            }
      

      Scenario:- Leaving bad inProgress_000X node ?
      Assume BKJM has created the inProgress_000X zNode and ZK is not available when trying to add the metadata. Now, inProgress_000X ends up with partial information.

      1. HDFS-3423.diff
        20 kB
        Ivan Kelly
      2. HDFS-3423.diff
        20 kB
        Ivan Kelly
      3. HDFS-3423.diff
        20 kB
        Ivan Kelly
      4. HDFS-3423.patch
        21 kB
        Uma Maheswara Rao G
      5. HDFS-3423.patch
        21 kB
        Uma Maheswara Rao G

        Issue Links

          Activity

          Rakesh R created issue -
          Hari Mankude made changes -
          Field Original Value New Value
          Project Hadoop HDFS [ 12310942 ] Bookkeeper [ 12311293 ]
          Key HDFS-3382 BOOKKEEPER-244
          Fix Version/s 0.24.0 [ 12317653 ]
          Ivan Kelly made changes -
          Assignee Ivan Kelly [ ikelly ]
          Ivan Kelly made changes -
          Project Bookkeeper [ 12311293 ] Hadoop HDFS [ 12310942 ]
          Key BOOKKEEPER-244 HDFS-3423
          Target Version/s 2.0.0, 3.0.0 [ 12320353, 12320356 ]
          Ivan Kelly made changes -
          Link This issue depends on HDFS-3058 [ HDFS-3058 ]
          Ivan Kelly made changes -
          Link This issue depends on HDFS-3058 [ HDFS-3058 ]
          Ivan Kelly made changes -
          Link This issue depends on HDFS-3058 [ HDFS-3058 ]
          Ivan Kelly made changes -
          Attachment HDFS-3423.diff [ 12527365 ]
          Ivan Kelly made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Uma Maheswara Rao G made changes -
          Parent HDFS-3399 [ 12554717 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]
          Ivan Kelly made changes -
          Attachment HDFS-3423.diff [ 12528085 ]
          Ivan Kelly made changes -
          Attachment HDFS-3423.diff [ 12530187 ]
          Uma Maheswara Rao G made changes -
          Attachment HDFS-3423.patch [ 12530358 ]
          Uma Maheswara Rao G made changes -
          Summary BookKeeperJournalManager: NN startup is failing, when tries to recoverUnfinalizedSegments() a bad inProgress_ ZNodes BKJM: NN startup is failing, when tries to recoverUnfinalizedSegments() a bad inProgress_ ZNodes
          Affects Version/s 3.0.0 [ 12320356 ]
          Uma Maheswara Rao G made changes -
          Attachment HDFS-3423.patch [ 12530371 ]
          Uma Maheswara Rao G made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.0.1-alpha [ 12321440 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue depends on HDFS-3058 [ HDFS-3058 ]
          Gavin made changes -
          Link This issue depends upon HDFS-3058 [ HDFS-3058 ]
          Allen Wittenauer made changes -
          Fix Version/s 3.0.0 [ 12320356 ]

            People

            • Assignee:
              Ivan Kelly
              Reporter:
              Rakesh R
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development