Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
Reviewed
Description
Hive on Tez disables auto-reduce and uses the maximum number of partitions when the minimum number of reducers, `estimated number * min.partition.factor`, is less than or equal to 1.
https://issues.apache.org/jira/browse/HIVE-14200
This can make Tez lose chance to optimize # of reducers especially in an environment with small `hive.tez.min.partition.factor` or big `hive.tez.max.partition.factor`.
Attachments
Issue Links
- relates to
-
HIVE-14200 Tez: disable auto-reducer parallelism when reducer-count * min.partition.factor < 1.0
- Resolved
- links to