Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1073 Simpler model for Namenode's fs Image and edit Logs
  3. HDFS-1800

Extend image checksumming to function with multiple fsimage files

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Edit log branch (HDFS-1073)
    • namenode
    • None
    • Reviewed

    Description

      HDFS-903 added the MD5 checksum of the fsimage to the VERSION file in each image directory. This allows it to verify that the FSImage didn't get corrupted or accidentally replaced on disk.

      With HDFS-1073, there may be multiple fsimage_N files in a storage directory corresponding to different checkpoints. So having a single MD5 in the VERSION file won't suffice. Instead we need to store an MD5 per image file.

      Attachments

        1. hdfs-1800-prelim.txt
          16 kB
          Todd Lipcon
        2. hdfs-1800.txt
          18 kB
          Todd Lipcon
        3. hdfs-1800.txt
          21 kB
          Todd Lipcon
        4. hdfs-1800.txt
          23 kB
          Todd Lipcon

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tlipcon Todd Lipcon Assign to me
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment