[PIG-4018] Schema validation fails with UNION ONSCHEMA

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.14.0
• Hadoop Flags: Reviewed
• Assignee: Travis Woodruff
• Reporter: Travis Woodruff
• Created: 18/Jun/14 16:44
• Updated: 21/Nov/14 05:58
• Resolved: 22/Jun/14 03:11

Description

When relations with differing schemas are unioned (using UNION ONSCHEMA), schema validation can fail with this exception:

org.apache.pig.impl.plan.PlanValidationException: Logical plan invalid state: invalid uid -1 in schema

This worked before the fix for PIG-3492.

The merged schema (from LOUnion.getSchema()) does not contain uids for columns not in the schema of the first input (uids are set to -1). This is because only the first input's schema is used for looking up "cached" uids.

Normally, this isn't a problem because UnionOnSchemaSetter comes along and fixes the missing fields.

However, when ImplicitSplitInsertVisitor is active, it is called before UnionOnSchemaSetter. ImplicitSplitInsertVisitor calls schemaResetter.visit(), which throws the validation exception because UnionOnSchemaSetter has not had a chance to create the missing fields (and thus uids are still -1 for these fields).