[IMPALA-12489] Error when scan kudu-1.17.0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 4.3.0
• Component/s: Backend, be
• Labels: scankudu
• Environment: centos7.9
• Epic Color: ghx-label-12
• Assignee: Unassigned
• Reporter: Pain Sun
• Created: 09/Oct/23 11:13
• Updated: 17/Apr/24 04:49
• Resolved: 17/Apr/24 04:46

Description

Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an empty string in primary key field.

sql:
select
    count(distinct thirdnick)
from
    member.qyexternaluserdetailinfo_new
where
    (
        mainshopnick = "xxx"
        and ownercorpid in ("xxx", "")
        and shoptype not in ("35", "56")
        and isDelete = 0
        and thirdnick != ""
        and thirdnick is not null
    );
 
error:ERROR: Unable to open scanner for node with id '1' for Kudu table 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such column: shopnick
 
If update sql like this:
select
    count(distinct thirdnick)
from
    member.qyexternaluserdetailinfo_new
where
    (
        mainshopnick = "xxx"
        and ownercorpid in ("xxx", "")
        and shopnick not in ('')
        and shoptype not in ("35", "56")
        and isDelete = 0
        and thirdnick != ""
        and thirdnick is not null
    );
no error.
 
this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
 
There is 1000000 items in this table ,280000 items where empty string.

 

create sql like this :
CREATE TABLE member.qyexternaluserdetailinfo_new (
  mainshopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  shopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  ownercorpid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  shoptype STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  clientid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  thirdnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  id BIGINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  receivermobile STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  thirdrealname STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  remark STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  createtime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  updatetime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  isdelete INT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION DEFAULT 0,
  buyernick STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
  PRIMARY KEY (
    mainshopnick,
    shopnick,
    ownercorpid,
    shoptype,
    clientid,
    thirdnick,
    id
  )
) PARTITION BY HASH (
  mainshopnick,
  shopnick,
  ownercorpid,
  shoptype,
  clientid,
  thirdnick,
  id
) PARTITIONS 10 STORED AS KUDU TBLPROPERTIES (
  'kudu.master_addresses' = '192.168.134.132,192.168.134.133,192.168.134.134',
  'kudu.num_tablet_replicas' = '1'
);

table schema like this:
--------------{}-------{}+{}-----{}{}---------{}{}--------{}{}------{}{}-----------{}{}-----------{}{}-----------------{}{}----------

name

type

comment

primary_key

key_unique

nullable

default_value

encoding

compression

block_size

--------------{}-------{}+{}-----{}{}---------{}{}--------{}{}------{}{}-----------{}{}-----------{}{}-----------------{}{}----------

mainshopnick	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
shopnick	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
ownercorpid	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
shoptype	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
clientid	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
thirdnick	string		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
id	bigint		true	true	false		AUTO_ENCODING	DEFAULT_COMPRESSION	0
receivermobile	string		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0
thirdrealname	string		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0
remark	string		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0
createtime	timestamp		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0
updatetime	timestamp		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0
isdelete	int		false		true	0	AUTO_ENCODING	DEFAULT_COMPRESSION	0
buyernick	string		false		true		AUTO_ENCODING	DEFAULT_COMPRESSION	0

--------------{}-------{}+{}-----{}{}---------{}{}--------{}{}------{}{}-----------{}{}-----------{}{}-----------------{}{}----------
 

Issue Links

• duplicates: KUDU-3518 node error when impala query