Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14552

Über-jira: WASB client phase II: performance and testing

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.1
    • Fix Version/s: 2.9.0, 3.0.0
    • Component/s: fs/azure
    • Labels:
      None

      Description

      Uber-JIRA for wasb:// phase II: the things we want for Hadoop 2.9.

      I think the focus is on performance, but looking at the tests there are some things we need there first: parallel test execution, move over to IT tests, other tuning.

      All patches must come with declarations of which azure endpoint they were tested against.

        Attachments

          Issue Links

          1.
          Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Sub-task Resolved Georgi Chalakov
          2.
          Upgrade azure-storage sdk version >5.4.0 Sub-task Resolved Rajesh Balamohan
          3.
          Update WASB driver to use the latest version (5.2.0) of SDK for Microsoft Azure Storage Clients Sub-task Closed Unassigned
          4.
          Remove service loader config file for wasb fs Sub-task Resolved Esfandiar Manii
          5.
          Add (parallelized) integration tests to hadoop-azure Sub-task Resolved Steve Loughran
          6.
          TestFileSystemOperationExceptionMessage to not rerun all of NativeAzureFileSystemBaseTest Sub-task Resolved Unassigned
          7.
          document how to run wasb tests in azure docs site/testing.md Sub-task Resolved Unassigned
          8.
          [WASB] the configured retry policy is not used for all storage operations. Sub-task Resolved Thomas Marquardt
          9.
          regression: Azure tests which capture logs failing with move to SLF4J Sub-task Resolved Steve Loughran
          10.
          wasb throws an exception if you try to create a file and there's no parent directory Sub-task Closed Esfandiar Manii
          11.
          Azure: Concurrent I/O does not work when secure.mode is enabled Sub-task Resolved Thomas Marquardt
          12.
          Azure: Reuse ObjectMapper Sub-task Resolved Mingliang Liu
          13.
          NPE in AzureNativeFileSystemStore.checkContainer() if StorageException lacks an error code Sub-task Resolved Steve Loughran
          14.
          error message in AzureNativeFileSystemStore.connectUsingAnonymousCredentials has too many spaces Sub-task Resolved Mingliang Liu
          15.
          wasb: add support for caching Authorization and SASKeys Sub-task Resolved Sivaguru Sankaridurg
          16.
          wasb: improve throughput by 34% when account limit exceeded Sub-task Resolved Thomas Marquardt
          17.
          TestWasbRemoteCallHelper failing Sub-task Resolved Esfandiar Manii
          18.
          Honoring sticky bit during Deletion when authorization is enabled in WASB Sub-task Resolved Varada Hemeswari
          19.
          WASB: delete recursive should not fail if a file is deleted Sub-task Resolved Thomas Marquardt
          20.
          PageBlobOutputStream to declare hflush/hsync StreamCapabilities for Sub-task Resolved Unassigned
          21.
          Add support for using container saskeys for all accesses Sub-task Resolved Sivaguru Sankaridurg
          22.
          Azure wasb: getFileStatus not making any auth checks Sub-task Resolved Sivaguru Sankaridurg
          23.
          Restrict Access to setPermission operation when authorization is enabled in WASB Sub-task Resolved Kannapiran Srinivasan
          24.
          Azure: POSIX permissions are taking effect in access() method even when authorization is enabled Sub-task Resolved Santhosh G Nayak
          25.
          Sticky bit implementation for rename() operation in Azure WASB Sub-task Resolved Varada Hemeswari
          26.
          ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run Sub-task Resolved Unassigned

            Activity

              People

              • Assignee:
                tmarquardt Thomas Marquardt
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: