[FLINK-1214] Prevent partitioning pushdown unless partitions fields match exactly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.8.0
• Component/s: API / DataSet
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 05/Nov/14 11:44
• Updated: 28/Feb/19 14:30
• Resolved: 06/Nov/14 13:02

Description

Consider an operation grouped on fields (A, B), followed by an operation grouped on field (A).

Right now, the optimizer can push down the partitioning on (A), which serves both operations (the first step locally still groups by A and B). This may however by a bad idea for the cases where the field A has a low cardinality, or the value distribution is skewed.

Since we cannot determine that robustly yet, I suggest to disable this optimization for now.