Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17694

abfs: Unable to use OAuth authentication at storage account level if the default authn type is Custom

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fs/azure, tools
    • Labels:
      None

      Description

      If we set the default auth type as Custom and then decided to use OAuth type for some select storage accounts then the fs initialization for storage accounts with Oauth type authn fails.

      Steps to recreate

      conf.set("fs.abfss.impl", "org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem")
      conf.set("fs.azure.account.auth.type", "Custom")
      conf.set("fs.azure.account.oauth.provider.type", "xxx.yyy.zzz.ADTokenAdaptee")
      conf.set("fs.azure.account.auth.type.abctest.dfs.core.windows.net", "OAuth")
      conf.set("fs.azure.account.oauth.provider.type.abctest.dfs.core.windows.net",
        "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider")
      
      val fs = FileSystem.get(
        new URI("abfss://conatinerxyz@abctest.dfs.core.windows.net/arion-scribe-de-dev"),
        conf)
      

      Error: java.lang.RuntimeException: class xxx.yyy.zzz.ADTokenAdaptee not org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider

      Cause:

      In AbfsConfiguration. getTokenProvider , after evaluating the auth type as OAuth, the program proceeds to get the implementing class using property `fs.azure.account.auth.type.abctest.dfs.core.windows.net`,  while doing so the first step is to get the default auth class (`fs.azure.account.oauth.provider.type`), which in our case is Custom. Here the problem is Default Auth class is CustomTokenProviderAdaptee implementation and not implementing  AccessTokenProvider.class, hence the program would fail.

      proposed solution:
      In the getClass function in AbfsConfiguration, we split the logic and not use the default value property

      public <U> Class<? extends U> getClass(String name, Class<? extends U> defaultValue, Class<U> xface) {
      
          Class<? extends U> klass = rawConfig.getClass(accountConf(name),
                  null, xface);
          
          if(klass!=null){
              return klass;
          }else{
              return rawConfig.getClass(name, defaultValue, xface);
          }
      }
      

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              arunravimv Arun Ravi M V
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: