[CARBONDATA-2017] Error occurs when loading multiple files

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Assignee: Manhua Jiang
• Reporter: Manhua Jiang
• Created: 10/Jan/18 14:04
• Updated: 19/Jan/18 12:33
• Resolved: 13/Jan/18 08:03

Description

Problem:
Carbon supports loading from multiple file paths at once, but we find that Carbon will throw an exception like "The input file does not exist" when loading multiple files on HDFS.

For example:
ex1: LOAD DATA INPATH '/data/source.csv,/data/source2.csv' INTO TABLE test_table
ex2: LOAD DATA INPATH 'hdfs://ha/data/source.csv,hdfs://ha/data/source2.csv' INTO TABLE test_table

ex1 will throw an exception saying that source2.csv does not exist.
ex2 will execute normally.

Solution:
We found that carbon takes the PATH as a whole and checks its prefix before spliting it into multiplt paths. So the problem will be solved when we do the prefix checking job for each path after spliting PATH into multiplt paths.

Issue Links

• links to: GitHub Pull Request #1787