[PIG-2054] Need to clarify globbing on command line vs in load statement

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: documentation
• Hadoop Flags: Reviewed
• Assignee: Corinne Chandel
• Reporter: Olga Natkovich
• Created: 09/May/11 18:51
• Updated: 04/Aug/11 00:34
• Resolved: 13/May/11 23:23

Description

We had several user reports saying that "globbing in Pig and Hadoop are not the same". They based this assertion on the fact that some patterns work from hadoop command line but would not work in Pig load statement.

Pig uses Hadoop globbing so the functionality is identical; however, when you run on command line, shell can be doing some of the substitution giving impression that things are different.

Example:

hadoop fs -ls /mydata/20110423{00,01,02,03,04,05,06,07,08,09,{10..23}}00//part - this works
LOAD '/mydata/20110423{00,01,02,03,04,05,06,07,08,09,{10..23}}00//part' - this does not

We should add a note to the description of globbing