Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3405

Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0, 2.0.5-alpha, 3.0.0-alpha1
    • 2.4.0
    • None
    • None

    Description

      As Todd points out in this comment, the current scheme for a checkpointing daemon to upload a merged fsimage file to an NN is to issue an HTTP get request to tell the target NN to issue another GET request back to the checkpointing daemon to retrieve the merged fsimage file. There's no fundamental reason the checkpointing daemon can't just use an HTTP POST or PUT to send back the merged fsimage file, rather than the double-GET scheme.

      Attachments

        1. HDFS-3405.patch
          73 kB
          Vinayakumar B
        2. HDFS-3405.patch
          73 kB
          Vinayakumar B
        3. HDFS-3405.patch
          71 kB
          Vinayakumar B
        4. HDFS-3405.patch
          67 kB
          Vinayakumar B
        5. HDFS-3405.patch
          67 kB
          Vinayakumar B
        6. HDFS-3405.patch
          41 kB
          Vinayakumar B
        7. HDFS-3405.patch
          42 kB
          Vinayakumar B
        8. HDFS-3405.patch
          46 kB
          Vinayakumar B
        9. HDFS-3405.patch
          46 kB
          Vinayakumar B
        10. HDFS-3405.patch
          45 kB
          Vinayakumar B
        11. HDFS-3405.patch
          44 kB
          Vinayakumar B
        12. HDFS-3405.patch
          44 kB
          Vinayakumar B
        13. HDFS-3405.patch
          43 kB
          Vinayakumar B
        14. HDFS-3405.patch
          42 kB
          Vinayakumar B
        15. HDFS-3405.patch
          43 kB
          Vinayakumar B
        16. HDFS-3405.patch
          43 kB
          Vinayakumar B
        17. HDFS-3405.patch
          42 kB
          Vinayakumar B
        18. HDFS-3405.patch
          39 kB
          Vinayakumar B

        Issue Links

        duplicates

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-5113 Implement POST method to upload the checkpoint

        • Major - Major loss of function.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-4301 2NN image transfer timeout problematic

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vinayakumarb Vinayakumar B
            atm Aaron Myers
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment