[HIVE-22560] ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.2
• Component/s: Query Planning
• Assignee: Unassigned
• Reporter: Aron Hamvas
• Created: 28/Nov/19 08:55
• Updated: 28/Nov/19 08:55

Description

If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in matching has NOT NULL constraint.

To reproduce the issue:

CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1 VARCHAR(100));
CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));

insert into simplecdcdelta values (123, 'abc');

merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num = d.cli_num
when matched then update set field1=d.field1
when not matched then insert values (d.cli_num, d.field1);

The NOT NULL constraint is then enforced in the Filter operator processing the output of the right outer join, where, of course, the not matched records have nulls in all columns.