Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.2.0
-
None
Description
The ability introduced by SPARK-14912, we can broadcast the parameters of the data source to the read and write operations, but if the user does not specify a specific parameter, the propagation operation will also be performed, which affects the performance has a greater impact, so we need to avoid broadcasting the full Hadoop parameters when the user does not specify a specific parameter
Attachments
Attachments
Issue Links
- duplicates
-
SPARK-44304 Broadcast operation is not required when no parameters are specified
- Closed
- is duplicated by
-
SPARK-44304 Broadcast operation is not required when no parameters are specified
- Closed
- relates to
-
SPARK-14912 Propagate data source options to Hadoop configurations
- Resolved
- links to