Details
-
Bug
-
Status: Patch Available
-
Minor
-
Resolution: Unresolved
-
2.2.0
-
None
-
None
Description
The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of either edge which cannot be represented by the ShuffleVertexManager presently.
Reducing the width based on the hashtable build side vs the streaming probe side have different consequences since there is no order of runtime between them.
Until the two parent vertices of the shuffle hash-join are related, this feature causes massive inconsistency of performance across runs.
For inner & semi joins, the hashtable side should have a higher priority than the streaming side and for left outer joins, the streaming side can over-take the hashtable side, being the more dominant factor in the final row-counts.
Until such priorities can be bubbled up into ShuffleVertexManager, this feature can be disabled.