[IMPALA-8557] Impala on ABFS failed with error "IllegalArgumentException: ABFS does not allow files or directories to end with a dot."

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.4.0
• Component/s: Backend
• Epic Color: ghx-label-8
• Assignee: Sahil Takiar
• Reporter: Eric Lin
• Created: 15/May/19 23:05
• Updated: 06/Nov/19 16:51
• Resolved: 06/Nov/19 16:51

Description

HDFS introduced below feature to stop users from creating a file that ends with "." on ABFS:
https://issues.apache.org/jira/browse/HADOOP-15860

As a result of this change, Impala now writes to ABFS fails with such error.

I can see that it generates temp file using this format "$0.$1.$2":
https://github.com/cloudera/Impala/blob/cdh6.2.0/be/src/exec/hdfs-table-sink.cc#L329

$2 is the file extension and will be empty if it is TEXT file format:
https://github.com/cloudera/Impala/blob/cdh6.2.0/be/src/exec/hdfs-text-table-writer.cc#L65

Since HADOOP-15860 was backported into CDH6.2, it is currently only affecting 6.2 and works in older versions.

There is no way to override this empty file extension so no workaround is possible, unless user choose another file format.

Issue Links

• is related to: IMPALA-7860 Tests use partition name that isn't supported on ABFS