Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15763

Über-JIRA: abfs phase II: Hadoop 3.3 features & fixes

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fs/azure
    • Labels:
      None

      Description

      ABFS phase II: address issues which surface in the field; tune things which need tuning, add more tests where appropriate. Improve docs, especially troubleshooting. Classpaths. The usual.

        Attachments

          Issue Links

          1.
          Review/implement ABFS support for the extra fs ops which some apps (HBase) expects Sub-task Open Unassigned  
          2.
          ABFS: Increase timeout of ITestAbfsReadWriteAndSeek Sub-task Patch Available Bilahari T H

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          3.
          ABFS to support a Delegation Token provider which marshalls current Oauth secrets Sub-task Open Unassigned  
          4.
          Add ABFS support for Async Scatter/Gather IO Sub-task Open Unassigned  
          5.
          ABFS tests to include terasort Sub-task Open Unassigned  
          6.
          ABFS VersionedFileStatus to declare that it isEncrypted() Sub-task Patch Available Masatake Iwasaki  
          7.
          ABFS: Rename operation, GetFileStatus before rename operation and throw exception on the driver side Sub-task Open Steve Loughran  
          8.
          ABFS: improve random read performance Sub-task Open Unassigned  
          9.
          abfs can't access storage account without password Sub-task Open Masatake Iwasaki  
          10.
          ABFS: Send ReadAhead feature utilization metrics to server Sub-task Open Sneha Vijayarajan  
          11.
          ABFS Adding an option to Always use Read Ahead, even for non sequential reads Sub-task Patch Available Saurabh  
          12.
          Add support for Filesystem#getFileChecksum in ABFS driver Sub-task Open Unassigned  
          13.
          ABFS: Have all external dependent module execution tracked with DurationInfo Sub-task Open Sneha Vijayarajan  
          14.
          abfs can't access storage account if soft delete is enabled Sub-task Open Unassigned  
          15.
          ABFS: Fix for OutofMemoryException from AbfsOutputStream Sub-task Open Bilahari T H  
          16.
          ABFS: Support single writer dirs Sub-task Open Billie Rinaldi

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10h 50m
          17.
          Clear abfs readahead requests on stream close Sub-task Open Unassigned  
          18.
          ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always Sub-task Reopened Steve Loughran  
          19.
          ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Sub-task Open Unassigned  
          20.
          ABFS: Random read perf improvement Sub-task Open Mukund Thakur

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h
          21.
          ABFS: Test testAbfsStreamOps timing out Sub-task Open Sneha Vijayarajan  
          22.
          ABFS: Testcase failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Sub-task Patch Available Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          23.
          ABFS: Add Identifiers to Client Request Header Sub-task Open Sumangala Patki

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 20m
          24.
          ABFS: Implementation for getContentSummary Sub-task Open Sumangala Patki  
          25.
          Pass exception to the caller in AbfsClient#appendSASTokenToQuery Sub-task Open Mukund Thakur  
          26.
          abfs ITestAzureBlobFileSystemCheckAccess test failure if test doesn't have oauth keys Sub-task Open Unassigned  
          27.
          ABFS: Suport for customer provided encrption key Sub-task Open Bilahari T H

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10h 10m
          28.
          ABFS test setup failing "The specified filesystem already exists" Sub-task Open Unassigned  
          29.
          ABFS: Disable throttling update for auth failures Sub-task Open Sumangala Patki

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h
          30.
          ABFS: Introduce Lease Operations with Append to provide single writer semantics Sub-task Open Sneha Varma  
          31.
          ABFS: Change default Readahead Queue Depth from num(processors) to const Sub-task Open Sumangala Patki

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          32.
          ABFS: Partially obfuscate SAS object IDs in Logs Sub-task Open Sumangala Patki

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          33.
          ABFS: Distcp contract test testDistCpWithIterator is timing out consistently Sub-task Open Unassigned  

            Activity

              People

              • Assignee:
                DanielZhou Da Zhou
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 106h 40m
                  106h 40m