[HADOOP-17725] Improve error message for token providers in ABFS - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Bulk Copy Attachments

Bulk Move Attachments

Voters

Stop watching

Watchers

Create sub-task

Convert to sub-task

Move

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.3.0
Fix Version/s: 3.3.2
Component/s: fs/azure, hadoop-thirdparty
Labels:
- pull-request-available

Target Version/s:

3.4.0, 3.3.5

Description

It would be good to improve error messages for token providers in ABFS. Currently, when a configuration key is not found or mistyped, the error is not very clear on what went wrong. It would be good to indicate that the key was required but not found in Hadoop configuration when creating a token provider.

For example, when running the following code:

import org.apache.hadoop.conf._
import org.apache.hadoop.fs._

val conf = new Configuration()
conf.set("fs.azure.account.auth.type", "OAuth")
conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
// conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net", "my-secret")
conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")

val path = new Path("abfss://container@my-account.dfs.core.windows.net/")
val fs = path.getFileSystem(conf)
fs.getFileStatus(path)

The following exception is thrown:

TokenAccessProviderException: Unable to load OAuth token provider class.
...
Caused by: UncheckedExecutionException: java.lang.NullPointerException: clientSecret
...
Caused by: NullPointerException: clientSecret

which does not tell what configuration key was not loaded.

IMHO, it would be good if the exception was something like this:

TokenAccessProviderException: Unable to load OAuth token provider class.
...
Caused by: ConfigurationPropertyNotFoundException: Configuration property fs.azure.account.oauth2.client.secret not found.