[PIG-3492] ColumnPrune dropping used column due to LogicalRelationalOperator.fixDuplicateUids changes not propagating

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.1, 0.12.1, 0.13.0
• Fix Version/s: 0.12.1
• Hadoop Flags: Reviewed
• Assignee: Koji Noguchi
• Reporter: Koji Noguchi
• Created: 01/Oct/13 18:57
• Updated: 15/Apr/14 20:45
• Resolved: 23/Oct/13 18:15

Description

I don't have a testcase I can upload at the moment, but here's my observation.

SplitFilter -> schemaResetter -> LOGenerate.getSchema -> LogicalRelationalOperator.fixDuplicateUids() creating a new UID but that UID is not propagated to the entire plan (since SplitFilter.reportChanges only returns subplan).

As a result, I am seeing ColumnPruning cutting off those used columns.

Issue Links

• is related to: PIG-3020 "Duplicate uid in schema" error when joining two relations derived from the same load statement
• is related to: PIG-3144 Erroneous map entry alias resolution leading to "Duplicate schema alias" errors