[PIG-1188] Padding nulls to the input tuple according to input schema

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.9.0
• Component/s: impl
• Hadoop Flags: Reviewed
• Release Note: If load statement specify schema, Pig will truncate/padding null to make sure the loaded data has exactly the same number of fields specified in load statement.
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 14/Jan/10 19:32
• Updated: 22/May/12 02:04
• Resolved: 09/Mar/11 22:58

Description

Currently, the number of fields in the input tuple is determined by the data. When we have schema, we should generate input data according to the schema, and padding nulls if necessary. Here is one example:

Pig script:

a = load '1.txt' as (a0, a1);
dump a;

Input file:

1       2
1       2       3
1

Current result:

(1,2)
(1,2,3)
(1)

Desired result:

(1,2)
(1,2)
(1, null)

Issue Links

• relates to: PIG-1131 Pig simple join does not work when it contains empty lines
• relates to: PIG-2661 Pig uses an extra job for loading data in Pigmix L9