Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8557

Impala on ABFS failed with error "IllegalArgumentException: ABFS does not allow files or directories to end with a dot."

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.2.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
      None
    • Epic Color:
      ghx-label-8

      Description

      HDFS introduced below feature to stop users from creating a file that ends with "." on ABFS:
      https://issues.apache.org/jira/browse/HADOOP-15860

      As a result of this change, Impala now writes to ABFS fails with such error.

      I can see that it generates temp file using this format "$0.$1.$2":
      https://github.com/cloudera/Impala/blob/cdh6.2.0/be/src/exec/hdfs-table-sink.cc#L329

      $2 is the file extension and will be empty if it is TEXT file format:
      https://github.com/cloudera/Impala/blob/cdh6.2.0/be/src/exec/hdfs-text-table-writer.cc#L65

      Since HADOOP-15860 was backported into CDH6.2, it is currently only affecting 6.2 and works in older versions.

      There is no way to override this empty file extension so no workaround is possible, unless user choose another file format.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ericlin Eric Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: