Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17089

WASB: Update azure-storage-java SDK

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.7.0, 2.8.0, 2.9.0, 3.0.0, 3.1.0, 3.2.0
    • Fix Version/s: 2.10.1, 3.3.1
    • Component/s: fs/azure
    • Labels:
      None
    • Release Note:
      Azure WASB bug fix that can cause list results to appear empty.

      Description

      WASB depends on the Azure Storage Java SDK.  There is a concurrency bug in the Azure Storage Java SDK that can cause the results of a list blobs operation to appear empty.  This causes the Filesystem listStatus and similar APIs to return empty results.  This has been seen in Spark work loads when jobs use more than one executor core. 

      See https://github.com/Azure/azure-storage-java/pull/546 for details on the bug in the Azure Storage SDK.

      This issue can cause data loss.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tmarquardt Thomas Marqardt
                Reporter:
                tmarquardt Thomas Marqardt
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: