Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17725

Improve error message for token providers in ABFS

    XMLWordPrintableJSON

Details

    Description

      It would be good to improve error messages for token providers in ABFS. Currently, when a configuration key is not found or mistyped, the error is not very clear on what went wrong. It would be good to indicate that the key was required but not found in Hadoop configuration when creating a token provider.

      For example, when running the following code:

      import org.apache.hadoop.conf._
      import org.apache.hadoop.fs._
      
      val conf = new Configuration()
      conf.set("fs.azure.account.auth.type", "OAuth")
      conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
      conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
      // conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net", "my-secret")
      conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")
      
      val path = new Path("abfss://container@my-account.dfs.core.windows.net/")
      val fs = path.getFileSystem(conf)
      fs.getFileStatus(path)

      The following exception is thrown:

      TokenAccessProviderException: Unable to load OAuth token provider class.
      ...
      Caused by: UncheckedExecutionException: java.lang.NullPointerException: clientSecret
      ...
      Caused by: NullPointerException: clientSecret 

      which does not tell what configuration key was not loaded.

       

      IMHO, it would be good if the exception was something like this:

      TokenAccessProviderException: Unable to load OAuth token provider class.
      ...
      Caused by: ConfigurationPropertyNotFoundException: Configuration property fs.azure.account.oauth2.client.secret not found. 

      Attachments

        Activity

          People

            vjasani Viraj Jasani
            ivan.sadikov Ivan Sadikov
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 8h
                8h