Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
Right now, the MapSideMergeValidator outright rejects any foreach that has a UDF in it:
private boolean isAcceptableForEachOp(Operator lo) throws LogicalToPhysicalTranslatorException { if (lo instanceof LOForEach) { OperatorPlan innerPlan = ((LOForEach) lo).getInnerPlan(); validateMapSideMerge(innerPlan.getSinks(), innerPlan); return !containsUDFs((LOForEach) lo); } else { return false; } }
There is a TODO for this later on in that same class (inside containsUDFs):
// TODO (dvryaboy): in the future we could relax this rule by tracing what fields // are being passed into the UDF, and only refusing if the UDF is working on the // join key. Transforms of other fields should be ok.
We should do the TODO and relax this requirement or just remove it altogether