[HIVE-1272] Add SymlinkTextInputFormat to Hive

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: Metastore, Query Processor
• Hadoop Flags: Reviewed
• Assignee: Guanghao Shen
• Reporter: Zheng Shao
• Created: 23/Mar/10 22:54
• Updated: 17/Dec/11 00:03
• Resolved: 01/Apr/10 18:35

Description

We'd like to add a symlink text input format so that we can specify the list of files for a table/partition based on the content of a text file.

For example, the location of the table is "/user/hive/mytable".
There is a file called "/user/hive/mytable/myfile.txt".
Inside the file, there are 2 lines, "/user/myname/textfile1.txt" and "/user/myname/textfile2.txt"

We can do:

CREATE TABLE mytable (...) STORED AS INPUTFORMAT 'org.apache.hadoop.hive.io.SymlinkTextInputFormat' LOCATION '/user/hive/mytable';
SELECT * FROM mytable;

which will return the content of the 2 files: "/user/myname/textfile1.txt" and "/user/myname/textfile2.txt"