Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12635

Adding Append API support for WASB

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.1
    • 2.8.0, 3.0.0-alpha1
    • fs/azure
    • None
    • Reviewed
    • Hide
      The Azure Blob Storage file system (WASB) now includes optional support for use of the append API by a single writer on a path. Please note that the implementation differs from the semantics of HDFS append. HDFS append internally guarantees that only a single writer may append to a path at a given time. WASB does not enforce this guarantee internally. Instead, the application must enforce access by a single writer, such as by running single-threaded or relying on some external locking mechanism to coordinate concurrent processes. Refer to the Azure Blob Storage documentation page for more details on enabling append in configuration.
      Show
      The Azure Blob Storage file system (WASB) now includes optional support for use of the append API by a single writer on a path. Please note that the implementation differs from the semantics of HDFS append. HDFS append internally guarantees that only a single writer may append to a path at a given time. WASB does not enforce this guarantee internally. Instead, the application must enforce access by a single writer, such as by running single-threaded or relying on some external locking mechanism to coordinate concurrent processes. Refer to the Azure Blob Storage documentation page for more details on enabling append in configuration.

    Description

      Currently the WASB implementation of the HDFS interface does not support Append API. This JIRA is added to design and implement the Append API support to WASB. The intended support for Append would only support a single writer.

      Attachments

        1. Append API.docx
          1.14 MB
          Dushyanth
        2. HADOOP-12635.001.patch
          49 kB
          Dushyanth
        3. HADOOP-12635.002.patch
          50 kB
          Dushyanth
        4. HADOOP-12635.003.patch
          51 kB
          Dushyanth
        5. HADOOP-12635.005.patch
          79 kB
          Dushyanth
        6. HADOOP-12635.006.patch
          79 kB
          Dushyanth
        7. HADOOP-12635.007.patch
          86 kB
          Dushyanth
        8. HADOOP-12635-004.patch
          51 kB
          Dushyanth

        Issue Links

          Activity

            People

              dchickabasapa Dushyanth
              dchickabasapa Dushyanth
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: