Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4141

Error when creating a partition that already exists in HMS

    Details

      Description

      Trying to add a partition which already exists in HMS but not in catalog throws an IllegalArgumentException. Steps to reproduce:

      impala> create table foo (a int) partitioned by (x int);
      impala> describe foo; <-- trigger a metadata load
      hive> alter table foo add partition (x = 1);
      impala> alter table foo add if not exists partition (x = 1);
      ERROR:
      IllegalArgumentException: Can not create a Path from a null string
      

      From the catalog.INFO we get the following stacktrace:

      I0915 14:22:00.215077  3710 jni-util.cc:169] java.lang.IllegalArgumentException: Can not create a Path from a null string
             	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)
             	at org.apache.hadoop.fs.Path.<init>(Path.java:135)
             	at com.cloudera.impala.catalog.HdfsTable.createPartition(HdfsTable.java:837)
             	at com.cloudera.impala.catalog.HdfsTable.createPartition(HdfsTable.java:792)
             	at com.cloudera.impala.service.CatalogOpExecutor.addHdfsPartition(CatalogOpExecutor.java:508)
             	at com.cloudera.impala.service.CatalogOpExecutor.alterTableAddPartition(CatalogOpExecutor.java:1707)
             	at com.cloudera.impala.service.CatalogOpExecutor.alterTable(CatalogOpExecutor.java:373)
             	at com.cloudera.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:239)
             	at com.cloudera.impala.service.JniCatalog.execDdl(JniCatalog.java:137)
      

        Activity

        Hide
        attilaj Attila Jeges added a comment -

        Change-Id: Iddbc951f2931f488f7048c9780260f6b49100750
        Reviewed-on: http://gerrit.cloudera.org:8080/4144
        Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
        Tested-by: Impala Public Jenkins

        M common/thrift/JniCatalog.thrift
        M fe/src/main/cup/sql-parser.cup
        M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java
        A fe/src/main/java/org/apache/impala/analysis/PartitionDef.java
        M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java
        M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java
        M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java
        M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
        M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
        M fe/src/test/java/org/apache/impala/analysis/AuthorizationTest.java
        M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
        M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
        M testdata/workloads/functional-query/queries/QueryTest/alter-table.test
        M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test
        M tests/common/impala_test_suite.py
        M tests/metadata/test_hms_integration.py
        M tests/metadata/test_refresh_partition.py
        17 files changed, 858 insertions, 207 deletions
        Approvals:
        Impala Public Jenkins: Verified
        Dimitris Tsirogiannis: Looks good to me, approved

        Show
        attilaj Attila Jeges added a comment - Change-Id: Iddbc951f2931f488f7048c9780260f6b49100750 Reviewed-on: http://gerrit.cloudera.org:8080/4144 Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com> Tested-by: Impala Public Jenkins — M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java A fe/src/main/java/org/apache/impala/analysis/PartitionDef.java M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M testdata/workloads/functional-query/queries/QueryTest/alter-table.test M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test M tests/common/impala_test_suite.py M tests/metadata/test_hms_integration.py M tests/metadata/test_refresh_partition.py 17 files changed, 858 insertions, 207 deletions Approvals: Impala Public Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved

          People

          • Assignee:
            attilaj Attila Jeges
            Reporter:
            dtsirogiannis Dimitris Tsirogiannis
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development