Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-265

Revisit append

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • None
    • None

    Description

      HADOOP-1700 and related issues have put a lot of efforts to provide the first implementation of append. However, append is such a complex feature. It turns out that there are issues that were initially seemed trivial but needs a careful design. This jira revisits append, aiming for a design and implementation supporting a semantics that are acceptable to its users.

      Attachments

        1. a.sh
          1 kB
          Tsz-wo Sze
        2. appendDesign.pdf
          408 kB
          Konstantin Shvachko
        3. appendDesign.pdf
          580 kB
          Hairong Kuang
        4. appendDesign1.pdf
          624 kB
          Hairong Kuang
        5. appendDesign2.pdf
          868 kB
          Hairong Kuang
        6. appendDesign3.pdf
          869 kB
          Hairong Kuang
        7. AppendSpec.pdf
          48 kB
          Hairong Kuang
        8. AppendTestPlan.html
          64 kB
          Konstantin I Boudnik
        9. AppendTestPlan.html
          65 kB
          Konstantin I Boudnik
        10. AppendTestPlan.html
          65 kB
          Konstantin I Boudnik
        11. AppendTestPlan.html
          65 kB
          Konstantin I Boudnik
        12. AppendTestPlan.html
          65 kB
          Konstantin I Boudnik
        13. AppendTestPlan.html
          63 kB
          Konstantin I Boudnik
        14. AppendTestPlan.html
          62 kB
          Konstantin I Boudnik
        15. AppendTestPlan.html
          62 kB
          Konstantin I Boudnik
        16. AppendTestPlan.html
          55 kB
          Konstantin I Boudnik
        17. TestPlanAppend.html
          50 kB
          Konstantin I Boudnik

        Issue Links

        incorporates

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-728 Create a comprehensive functional test for append

        • Major - Major loss of function.
        • Closed
        is related to

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-6239 Command-line for append

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. HDFS-744 Support hsync in HDFS

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-1831 HDFS equivalent of HADOOP-7223 changes to handle FileContext createFlag combinations

        • Major - Major loss of function.
        • Closed
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-588 Fix TestFiDataTransferProtocol and TestAppend2 failures in append branch.

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-1060 Append/flush should support concurrent "tailer" use case

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-195 Need to handle access token expiration when re-establishing the pipeline for dfs write

        • Major - Major loss of function.
        • Closed

        New Feature - A new feature of the product, which has yet to be developed. HDFS-1606 Provide a stronger data guarantee in the write pipeline

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-4359 Access Token: Support for data access authorization checking on DataNodes

        • Major - Major loss of function.
        • Closed

        Test - A new unit, integration or system test. HDFS-714 Create fault injection test for the new pipeline close

        • Major - Major loss of function.
        • Closed

        Test - A new unit, integration or system test. HDFS-451 Test DataTransferProtocol with fault injection

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed
        supercedes

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-1700 Append to files in HDFS

        • Major - Major loss of function.
        • Closed
        There are no Sub-Tasks for this issue.

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hairong Hairong Kuang
            hairong Hairong Kuang
            Votes:
            5 Vote for this issue
            Watchers:
            54 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment