[PIG-1513] Pig doesn't handle empty input directory

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Hadoop Flags: Reviewed
• Assignee: Richard Ding
• Reporter: Richard Ding
• Created: 22/Jul/10 18:49
• Updated: 24/Feb/17 16:01
• Resolved: 30/Jul/10 16:56

Description

The following script

A = load 'input';
B = load 'emptydir';
C = join B by $0, A by $0 using 'skewed';
store C into 'output';

fails with "ERROR: java.lang.RuntimeException: Empty samples file';

In this case, the sample job has 0 maps. Pig doesn't expect this and fails .

For merge join the script

The merge join script

A = load 'input';
B = load 'emptydir';
C = join A by $0, B by $0 using 'merge';
store C into 'output';

the sample job again has 0 maps and the script fails with " ERROR 2176: Error processing right input during merge join".

But if we change the join order:

A = load 'input';
B = load 'emptydir';
C = join B by $0, A by $0 using 'merge';
store C into 'output';

The second job (merge) now has 0 maps and 0 reduces. And it generates an empty 'output' directory.

Order by on empty directory works fine and generates empty part files.

Issue Links

• is related to: PIG-5140 fix TestEmptyInputDir unit test failure after PIG-5132