[SPARK-49798] Fix inaccurate documentation of RuntimeConfig.get

• Type: Documentation
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Xi Lyu
• Created: 26/Sep/24 13:32
• Updated: 26/Sep/24 13:38

Description

The existing documentation of `RuntimeConfig.get()` is misleading:

• `get(key: String)` method will not throw any exception if the key is not set as long as the config entry has a default value, instead, they will just return the `defaultValue` of the `ConfigEntry`. An `NoSuchElementException` will only be thrown if there is no default value for the config entry.
• `get(key: String, default: String)` method will ignore the `defaultValue` of its `ConfigEntry`, and return the given param `default` if unset. 
• `getOption(key: String)` method will return the `defaultValue` of its `ConfigEntry` if the config is not set.

 

An example:

This line makes sure the config is not set.
```
scala> spark.conf.unset("spark.sql.session.timeZone") 
```

The following code returns `Etc/UTC`, which doesn't throw any exception.
```
scala> spark.conf.get("spark.sql.session.timeZone") 
res2: String = "Etc/UTC"
```

The following code returns `Europe/Berlin`, ignoring the default value. However, the documentation only says it returns the value, without mentioning ignoring the default value of the entry when the config is not explicitly set.
```
scala> spark.conf.get("spark.sql.session.timeZone", "Europe/Berlin") 
res3: String = "Europe/Berlin"
```

The incorrect documentation is likely to mislead users to weird behaviours if they rely on the documentation.