Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3405

Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0, 2.0.5-alpha, 3.0.0-alpha1
    • 2.4.0
    • None
    • None

    Description

      As Todd points out in this comment, the current scheme for a checkpointing daemon to upload a merged fsimage file to an NN is to issue an HTTP get request to tell the target NN to issue another GET request back to the checkpointing daemon to retrieve the merged fsimage file. There's no fundamental reason the checkpointing daemon can't just use an HTTP POST or PUT to send back the merged fsimage file, rather than the double-GET scheme.

      Attachments

        1. HDFS-3405.patch
          39 kB
          Vinayakumar B
        2. HDFS-3405.patch
          42 kB
          Vinayakumar B
        3. HDFS-3405.patch
          43 kB
          Vinayakumar B
        4. HDFS-3405.patch
          43 kB
          Vinayakumar B
        5. HDFS-3405.patch
          42 kB
          Vinayakumar B
        6. HDFS-3405.patch
          43 kB
          Vinayakumar B
        7. HDFS-3405.patch
          44 kB
          Vinayakumar B
        8. HDFS-3405.patch
          44 kB
          Vinayakumar B
        9. HDFS-3405.patch
          45 kB
          Vinayakumar B
        10. HDFS-3405.patch
          46 kB
          Vinayakumar B
        11. HDFS-3405.patch
          46 kB
          Vinayakumar B
        12. HDFS-3405.patch
          42 kB
          Vinayakumar B
        13. HDFS-3405.patch
          41 kB
          Vinayakumar B
        14. HDFS-3405.patch
          67 kB
          Vinayakumar B
        15. HDFS-3405.patch
          67 kB
          Vinayakumar B
        16. HDFS-3405.patch
          71 kB
          Vinayakumar B
        17. HDFS-3405.patch
          73 kB
          Vinayakumar B
        18. HDFS-3405.patch
          73 kB
          Vinayakumar B

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-5113 Implement POST method to upload the checkpoint

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-4301 2NN image transfer timeout problematic

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              vinayakumarb Vinayakumar B
              atm Aaron Myers
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: