[HDFS-3423] BKJM: NN startup is failing, when tries to recoverUnfinalizedSegments() a bad inProgress_ ZNodes

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.0.2-alpha
• Target Version/s: 2.0.0-alpha
• Hadoop Flags: Reviewed
• Assignee: Ivan Kelly
• Reporter: Rakesh Radhakrishnan
• Created: 08/May/12 07:02
• Updated: 12/May/16 18:12
• Resolved: 31/May/12 18:28

Description

Say, the InProgress_000X node is corrupted due to not writing the data(version, ledgerId, firstTxId) to this inProgress_000X znode. Namenode startup has the logic to recover all the unfinalized segments, here will try to read the segment and getting shutdown.

EditLogLedgerMetadata.java:

static EditLogLedgerMetadata read(ZooKeeper zkc, String path)
      throws IOException, KeeperException.NoNodeException  {
      byte[] data = zkc.getData(path, false, null);
      String[] parts = new String(data).split(";");
      if (parts.length == 3)
         ....reading inprogress metadata
      else if (parts.length == 4)
         ....reading inprogress metadata
      else
        throw new IOException("Invalid ledger entry, "
                              + new String(data));
      }

Scenario:- Leaving bad inProgress_000X node ?
Assume BKJM has created the inProgress_000X zNode and ZK is not available when trying to add the metadata. Now, inProgress_000X ends up with partial information.

Issue Links

• depends upon: HDFS-3058 HA: Bring BookKeeperJournalManager up to date with HA changes