[PIG-4685] SecondaryKeyOptimizerTez does not optimize cogroup

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Rohini Palaniswamy
• Created: 22/Sep/15 22:25
• Updated: 22/Sep/15 22:25

Description

In PIG-4683, daijy pointed out that SecondaryKeyOptimizerTez does not optimize cogroup.

// Current code does not handle more than one predecessors
        // even though it is possible. The problem is when we
        // process the first predecessor, we remove the foreach inner
        // operators from the reduce side, and the second predecessor
        // cannot see them
        if (predecessors.size()>1) {
            return;
        }

It is a tricky case and we need to see how to fix it.