[SPARK-33451] change 'spark.sql.adaptive.skewedPartitionThresholdInBytes' to 'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes'

• Type: Documentation
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0, 3.0.1
• Fix Version/s: 3.0.2, 3.1.0
• Component/s: Documentation
• Target Version/s: 3.0.0, 3.0.1
• Assignee: aof
• Reporter: aof
• Created: 14/Nov/20 13:55
• Updated: 12/Dec/22 18:11
• Resolved: 16/Nov/20 01:35

Description

In the 'Optimizing Skew Join' section of the following two pages:

1. https://spark.apache.org/docs/3.0.0/sql-performance-tuning.html
2. https://spark.apache.org/docs/3.0.1/sql-performance-tuning.html

The configuration 'spark.sql.adaptive.skewedPartitionThresholdInBytes' should be changed to 'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes', The former is missing the 'skewJoin'.

Issue Links

• links to: [Github] Pull Request #30376 (aof00)