Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
tez-branch
-
None
Description
For tez to compute the number of reducers, we should be walking the operator tree in a topological fashion so that the reducers down the tree get the estimate from all parents. However, the current walkers in hive only walk the operator tree in a depth-first fashion. We need to add a new walker for the topological walk. Also, since information about the parent operators needs to be propagated on a per parent basis, we need to retain some context across operators to be passed to the child which the walker will co-ordinate.
NO PRECOMMIT TESTS (this is wip for the tez branch)