Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4996

ClientProtocol#metaSave can be made idempotent by overwriting the output file instead of appending to it

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.1.0-beta, 3.0.0-alpha1
    • 2.1.0-beta
    • namenode
    • None
    • Reviewed
    • The dfsadmin -metasave command has been changed to overwrite the output file. Previously, this command would append to the output file if it already existed.

    Description

      ClientProtocol#metaSave opens its output file for append. This prevents the operation from being idempotent, because retries can cause unpredictable side effects (single copy of the output vs. multiple copies). As discussed on HDFS-4974, the operation could be made idempotent by changing this to open the file for overwrite instead of append.

      Attachments

        1. HDFS-4996.1.patch
          6 kB
          Chris Nauroth
        2. HDFS-4996.2.patch
          8 kB
          Chris Nauroth

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cnauroth Chris Nauroth
            cnauroth Chris Nauroth
            Votes:
            0 Vote for this issue
            Watchers:
            5 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment