Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19092 ABFS phase 4: post Hadoop 3.4.0 features
  3. HADOOP-19208

ABFS: Fixing logic to determine HNS nature of account to avoid extra getAcl() calls

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • fs/azure
    • None

    Description

      ABFS driver needs to know the type of account being used. It relies on the user to inform the account type using the config `fs.azure.account.hns.enabled`.
      If not configured, driver makes a getAcl call to determine the account type.

      Expectation is getAcl() will fail with 400 Bad Request if made on the FNS Account.
      For any other case including 200, 404 it will indicate account is HNS.

      Today, when determining this, the logic only checks status code to be either 200 or 400. In case of 404, nothing is inferred, and this leads to more getAcl again and again until 200 or 400 comes.

      Fix is to update the logic such that if getAcl() fails with 400, it is FNS account. For all other cases it will be an HNS account. In case of throttling, if all retries are exhausted, FS init itself will fail.

      This is also to fix a test case failing on trunk. testGetAclCallOnHnsConfigAbsence(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate)

      Attachments

        Activity

          People

            anujmodi2021 Anuj Modi
            anujmodi2021 Anuj Modi
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: