[PIG-1071] Support comma separated file/directory names in load statements

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Hadoop Flags: Reviewed
• Assignee: Richard Ding
• Reporter: Richard Ding
• Created: 04/Nov/09 01:10
• Updated: 24/Mar/10 22:15
• Resolved: 06/Nov/09 05:12

Description

Currently Pig Latin support following LOAD syntax:

LOAD 'data' [USING loader function] [AS schema];      

where data is the name of the file or directory, including files specified with Hadoop-supported globing syntax. This name is passed to the loader function.

This feature is to support loaders that can load multiple files from different directories and allows users to pass in the file names in a comma separated string.

For example, these will be valid load statements:

LOAD '/usr/pig/test1/a,/usr/pig/test2/b' USING someloader()';

and

LOAD '/usr/pig/test1/{a,c},/usr/pig/test2/b' USING someloader();

This comma separated string is passed to the loader.