Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12635

Adding Append API support for WASB

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.1
    • 2.8.0, 3.0.0-alpha1
    • fs/azure
    • None
    • Reviewed
    • Hide
      The Azure Blob Storage file system (WASB) now includes optional support for use of the append API by a single writer on a path. Please note that the implementation differs from the semantics of HDFS append. HDFS append internally guarantees that only a single writer may append to a path at a given time. WASB does not enforce this guarantee internally. Instead, the application must enforce access by a single writer, such as by running single-threaded or relying on some external locking mechanism to coordinate concurrent processes. Refer to the Azure Blob Storage documentation page for more details on enabling append in configuration.
      Show
      The Azure Blob Storage file system (WASB) now includes optional support for use of the append API by a single writer on a path. Please note that the implementation differs from the semantics of HDFS append. HDFS append internally guarantees that only a single writer may append to a path at a given time. WASB does not enforce this guarantee internally. Instead, the application must enforce access by a single writer, such as by running single-threaded or relying on some external locking mechanism to coordinate concurrent processes. Refer to the Azure Blob Storage documentation page for more details on enabling append in configuration.

    Description

      Currently the WASB implementation of the HDFS interface does not support Append API. This JIRA is added to design and implement the Append API support to WASB. The intended support for Append would only support a single writer.

      Attachments

        1. HADOOP-12635-004.patch
          51 kB
          Dushyanth
        2. HADOOP-12635.007.patch
          86 kB
          Dushyanth
        3. HADOOP-12635.006.patch
          79 kB
          Dushyanth
        4. HADOOP-12635.005.patch
          79 kB
          Dushyanth
        5. HADOOP-12635.003.patch
          51 kB
          Dushyanth
        6. HADOOP-12635.002.patch
          50 kB
          Dushyanth
        7. HADOOP-12635.001.patch
          49 kB
          Dushyanth
        8. Append API.docx
          1.14 MB
          Dushyanth

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dchickabasapa Dushyanth Assign to me
            dchickabasapa Dushyanth
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment