[BEAM-8307] NPE in Calcite dialect when input PCollection has logical type in schema, from JdbcIO Transform

• Type: Bug
• Status: Resolved
• Priority: P3
• Resolution: Duplicate
• Affects Version/s: 2.15.0
• Fix Version/s: Missing
• Component/s: dsl-sql, io-java-jdbc
• Assignee: Unassigned
• Reporter: Rahul Patwari
• Created: 24/Sep/19 09:09
• Updated: 13/Jan/22 21:51
• Resolved: 11/May/21 02:08

Description

An exception is thrown when SqlTransform is used with JdbcIO.

Schema of the Source Table:

CREATE TABLE actor (
actor_id SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT,
first_name VARCHAR(45) NOT NULL,
last_name VARCHAR(45) NOT NULL,
last_update TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (actor_id),
KEY idx_actor_last_name (last_name)
)ENGINE=InnoDB DEFAULT CHARSET=utf8; 

 

Many of the Jdbc Types are converted to Logical_Types in Beam.

Refer: https://github.com/apache/beam/blob/113461a4275eb28a101e416897ccd607aa51f83f/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/SchemaUtil.java#L102

The initial observation is that this code: https://github.com/apache/beam/blob/113461a4275eb28a101e416897ccd607aa51f83f/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtils.java#L207 is returning null.

 

PFA exception_sqlTransform.txt

Issue Links

• duplicates: BEAM-8888 BeamSQL does not support LogicalType
• links to: GitHub Pull Request #11581