[SPARK-23425] load data for hdfs file path with wild card usage is not working properly - ASF JIRA

XML

Word

Printable

JSON

Docs Text:

Hide
Release notes:

Wildcard symbols {{*}} and {{?}} can now be used in SQL paths when loading data, e.g.:

LOAD DATA INPATH 'hdfs://hacluster/user/ext*'
LOAD DATA INPATH 'hdfs://hacluster/user/???/data'

Where these characters are used literally in paths, they must be escaped with a backslash.

Wildcards can be used in the folder level of a local File system in Load command from now.

e.g. LOAD DATA LOCAL INPATH 'tmp/folder*/

Now onward normal Space convention can be used in folder/file names (e.g file Name.csv), Older versions space in folder/file names has been represented using '%20'(e.g. myFile%20Name).

#

Show
Release notes: Wildcard symbols {{*}} and {{?}} can now be used in SQL paths when loading data, e.g.: LOAD DATA INPATH ' hdfs://hacluster/user/ext*' LOAD DATA INPATH ' hdfs://hacluster/user/???/data' Where these characters are used literally in paths, they must be escaped with a backslash. Wildcards can be used in the folder level of a local File system in Load command from now. e.g. LOAD DATA LOCAL INPATH 'tmp/folder*/ Now onward normal Space convention can be used in folder/file names (e.g file Name.csv), Older versions space in folder/file names has been represented using '%20'(e.g. myFile%20Name). #

load data command for loading data from non local file paths by using wild card strings lke * are not working

eg:

"load data inpath 'hdfs://hacluster/user/ext* into table t1"

Getting Analysis excepton while executing this query

causes

SPARK-25738 LOAD DATA INPATH doesn't work if hdfs conf includes port

links to

[Github] Pull Request #20611 (sujith71955)