Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1073 Simpler model for Namenode's fs Image and edit Logs
  3. HDFS-1801

Remove use of timestamps to identify checkpoints and logs

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Edit log branch (HDFS-1073)
    • namenode
    • None
    • Reviewed

    Description

      Currently, the NameNode validates checkpoint uploads by using timestamps associated with checkpoints and edit logs. However, now that we have transaction IDs that uniquely identify each point in time in the history of a namespace, it is more robust to simply use transaction IDs to identify images and edits.

      This JIRA is to remove the use of editsTime and checkpointTime and replace it with:

      • lastCheckpointTxId - the highest transaction ID reflected in the most recently saved fsimage file
      • lastLogRollTxId - the highest transaction ID in edits when rollFsImage was called by the checkpointing node.

      Attachments

        1. hdfs-1801.txt
          41 kB
          Todd Lipcon
        2. hdfs-1801.txt
          41 kB
          Todd Lipcon
        3. hdfs-1801.txt
          39 kB
          Todd Lipcon
        4. hdfs-1801.txt
          32 kB
          Todd Lipcon
        5. hdfs-1801.txt
          31 kB
          Todd Lipcon

        Issue Links

          Activity

            People

              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: