[HADOOP-17725] Improve error message for token providers in ABFS

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.2
• Component/s: fs/azure, hadoop-thirdparty
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.5
• Assignee: Viraj Jasani
• Reporter: Ivan Sadikov
• Created: 21/May/21 11:07
• Updated: 28/Nov/22 16:54
• Resolved: 04/May/22 10:41

Description

It would be good to improve error messages for token providers in ABFS. Currently, when a configuration key is not found or mistyped, the error is not very clear on what went wrong. It would be good to indicate that the key was required but not found in Hadoop configuration when creating a token provider.

For example, when running the following code:

import org.apache.hadoop.conf._
import org.apache.hadoop.fs._

val conf = new Configuration()
conf.set("fs.azure.account.auth.type", "OAuth")
conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
// conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net", "my-secret")
conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")

val path = new Path("abfss://[e-mail]/")
val fs = path.getFileSystem(conf)
fs.getFileStatus(path)

The following exception is thrown:

TokenAccessProviderException: Unable to load OAuth token provider class.
...
Caused by: UncheckedExecutionException: java.lang.NullPointerException: clientSecret
...
Caused by: NullPointerException: clientSecret 

which does not tell what configuration key was not loaded.

 

IMHO, it would be good if the exception was something like this:

TokenAccessProviderException: Unable to load OAuth token provider class.
...
Caused by: ConfigurationPropertyNotFoundException: Configuration property fs.azure.account.oauth2.client.secret not found. 

Issue Links

• causes: HADOOP-18542 Azure Token provider requires tenant and client IDs despite being optional
• links to: GitHub Pull Request #3041
• links to: GitHub Pull Request #3788
• links to: GitHub Pull Request #4262