Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13189

Standby NameNode should roll active edit log when checkpointing

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: namenode
    • Labels:
      None

      Description

      When the SBN is doing checkpointing, it will hold the cpLock. In the current implementation of edit log tailer thread, it will first check and roll active edit log, and then tail and apply edits. In the case of checkpointing, it will be blocked on the cpLock and will not roll the edit log.

      It seems there is no dependency between the edit log roll and tailing edits, so a better may be to do these in separate threads. This will be helpful for people who uses the observer feature without in-progress edit log tailing.

      An alternative is to configure dfs.namenode.edit.log.autoroll.multiplier.threshold and dfs.namenode.edit.log.autoroll.check.interval.ms to let ANN roll its own log more frequently in case SBN is stuck on the lock.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                csun Chao Sun
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: