[HDFS-16836] StandbyCheckpointer can still trigger rollback fs image after RU is finalized

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.5
• Fix Version/s: 3.4.0, 3.3.5
• Component/s: hdfs
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.5
• Hadoop Flags: Reviewed
• Assignee: Lei Yang
• Reporter: Lei Yang
• Created: 09/Nov/22 23:17
• Updated: 28/Jan/24 07:53
• Resolved: 15/Nov/22 23:13

Description

StandbyCheckpointer trigger rollback fsimage when RU is started.

When ru is started, a flag (needRollbackImage) was set to true during edit log replay.

And it only gets reset to false when doCheckpoint() succeeded.

Think about following scenario:

1. Start RU, needRollbackImage is set to true.
2. doCheckpoint() failed.
3. RU is finalized.
4. namesystem.getFSImage().hasRollbackFSImage() is always false since rollback image cannot be generated once RU is over.
5. needRollbackImage was never set to false.
6. Checkpoints threshold(1m txns) and period(1hr) are not honored.

StandbyCheckpointer:
void doWork() {
 ....
  doCheckpoint();

  // reset needRollbackCheckpoint to false only when we finish a ckpt
  // for rollback image
  if (needRollbackCheckpoint
      && namesystem.getFSImage().hasRollbackFSImage()) {
    namesystem.setCreatedRollbackImages(true);
    namesystem.setNeedRollbackFsImage(false);
  }
  lastCheckpointTime = now;
} 

Issue Links

• links to: GitHub Pull Request #5135