Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17089

WASB: Update azure-storage-java SDK

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.7.0, 2.8.0, 2.9.0, 3.0.0, 3.1.0, 3.2.0
    • Fix Version/s: 2.10.1, 3.3.1
    • Component/s: fs/azure
    • Labels:
      None
    • Release Note:
      Azure WASB bug fix that can cause list results to appear empty.

      Description

      WASB depends on the Azure Storage Java SDK.  There is a concurrency bug in the Azure Storage Java SDK that can cause the results of a list blobs operation to appear empty.  This causes the Filesystem listStatus and similar APIs to return empty results.  This has been seen in Spark work loads when jobs use more than one executor core. 

      See https://github.com/Azure/azure-storage-java/pull/546 for details on the bug in the Azure Storage SDK.

      This issue can cause data loss.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              tmarquardt Thomas Marqardt
              Reporter:
              tmarquardt Thomas Marqardt

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment