[IMPALA-13244] Timestamp partition error in catalogd when insert data into iceberg table

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 4.4.0
• Component/s: Catalog
• Environment: centos7.9
• Epic Color: ghx-label-12
• Assignee: Unassigned
• Reporter: Pain Sun
• Created: 19/Jul/24 02:19
• Updated: 22/Jul/24 01:30

Description

create table sql like this:
      
CREATE TABLE test111.table1 (
    xxx STRING,
    code STRING,
    updatetime TIMESTAMP
) PARTITIONED BY spec(
    month(updatetime),
    bucket(10, code),
    bucket(10, xxx)
) STORED AS ICEBERG TBLPROPERTIES(
    'iceberg.catalog' = 'hadoop.catalog',
    'iceberg.catalog_location' = '/impalatable',
    'iceberg.table_identifier' = 'middle.table1',
    'write.metadata.previous-versions-max' = '3',
    'write.metadata.delete-after-commit.enabled' = 'true',
    'commit.manifest.min-count-to-merge' = '3',
    'commit.manifest-merge.enabled' = 'true',
    'format-version' = '1'
);
 
 
 
then insert data into this table like this:
insert into
    test111.table1 (
        xxx,
        code,
        updatetime
    )
select
    'm1' as xxx,
    'c1' as code,
    '2024-07-17 13:44:01' as updatetime;
Catalogd error like this :
E0719 09:50:57.458815 126128 JniUtil.java:183] 964d388b63170b6b:7c6e06c200000000] Error in Update catalog for test111.table1. Time spent: 6ms
I0719 09:50:57.459015 126128 jni-util.cc:302] 964d388b63170b6b:7c6e06c200000000] java.lang.IllegalStateException
        at com.google.common.base.Preconditions.checkState(Preconditions.java:496)
        at org.apache.impala.util.IcebergUtil.parseMonthToTransformMonth(IcebergUtil.java:882)
        at org.apache.impala.util.IcebergUtil.getPartitionValue(IcebergUtil.java:826)
        at org.apache.impala.util.IcebergUtil.partitionDataFromDataFile(IcebergUtil.java:800)
        at org.apache.impala.service.IcebergCatalogOpExecutor.createDataFile(IcebergCatalogOpExecutor.java:445)
        at org.apache.impala.service.IcebergCatalogOpExecutor.appendFiles(IcebergCatalogOpExecutor.java:487)
        at org.apache.impala.service.IcebergCatalogOpExecutor.execute(IcebergCatalogOpExecutor.java:366)
        at org.apache.impala.service.CatalogOpExecutor.updateCatalogImpl(CatalogOpExecutor.java:7443)
        at org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:7180)
        at org.apache.impala.service.JniCatalog.lambda$updateCatalog$15(JniCatalog.java:504)
        at org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
        at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
        at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:245)
        at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:259)
        at org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:503)
I0719 09:50:57.459033 126128 status.cc:129] 964d388b63170b6b:7c6e06c200000000] IllegalStateException: null
    @          0x10546b4
    @          0x1b94d34
    @          0x10040ab
    @           0xfa1c27
    @           0xf61f84
    @           0xf4acc3
    @           0xf5278b
    @          0x14486aa
    @          0x143b0fa
    @          0x1c78d39
    @          0x1c79fd1
    @          0x256da47
    @     0x7fabd2eb8ea5
    @     0x7fabcfe939fd
E0719 09:50:57.459059 126128 catalog-server.cc:324] 964d388b63170b6b:7c6e06c200000000] IllegalStateException: null
 
but spark insert success.
 
versions :
impala:  4.4.0
jar in impala:  iceberg-api-1.3.1.7.2.18.0-369.jar
spark:  3.3.4
iceberg:  apache 1.3.1
iceberg-spark jar:  iceberg-spark-runtime-3.3_2.12-1.3.1.jar