Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17725

Improve error message for token providers in ABFS

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersStop watchingWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      It would be good to improve error messages for token providers in ABFS. Currently, when a configuration key is not found or mistyped, the error is not very clear on what went wrong. It would be good to indicate that the key was required but not found in Hadoop configuration when creating a token provider.

      For example, when running the following code:

      import org.apache.hadoop.conf._
      import org.apache.hadoop.fs._
      
      val conf = new Configuration()
      conf.set("fs.azure.account.auth.type", "OAuth")
      conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
      conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
      // conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net", "my-secret")
      conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")
      
      val path = new Path("abfss://container@my-account.dfs.core.windows.net/")
      val fs = path.getFileSystem(conf)
      fs.getFileStatus(path)

      The following exception is thrown:

      TokenAccessProviderException: Unable to load OAuth token provider class.
      ...
      Caused by: UncheckedExecutionException: java.lang.NullPointerException: clientSecret
      ...
      Caused by: NullPointerException: clientSecret 

      which does not tell what configuration key was not loaded.

       

      IMHO, it would be good if the exception was something like this:

      TokenAccessProviderException: Unable to load OAuth token provider class.
      ...
      Caused by: ConfigurationPropertyNotFoundException: Configuration property fs.azure.account.oauth2.client.secret not found. 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vjasani Viraj Jasani Assign to me
            ivan.sadikov Ivan Sadikov
            Votes:
            0 Vote for this issue
            Watchers:
            8 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 8h
              8h

              Slack

                Issue deployment