[PHOENIX-4645] PhoenixStorageHandler doesn't handle correctly data/timestamp in push down predicate when engine is tez.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.14.0
• Fix Version/s: 4.14.0
• Labels: HivePhoenix
• Assignee: Sergey Soldatov
• Reporter: Sergey Soldatov
• Created: 08/Mar/18 00:27
• Updated: 26/Jul/18 01:15
• Resolved: 24/Apr/18 06:41

Description

DDLs:

CREATE TABLE TEST_PHOENIX
(
PART_ID BIGINT NOT NULL,
COMMIT_TIMESTAMP TIMESTAMP,
CONSTRAINT pk PRIMARY KEY (PART_ID)
)
SALT_BUCKETS=9;
CREATE EXTERNAL TABLE TEST_HIVE
(
PART_ID BIGINT,
SOURCEDB_COMMIT_TIMESTAMP TIMESTAMP
)
STORED BY 'org.apache.phoenix.hive.PhoenixStorageHandler'
TBLPROPERTIES
(
"phoenix.table.name" = "TEST_PHOENIX",
"phoenix.zookeeper.quorum" = "localhost",
"phoenix.zookeeper.znode.parent" = "/hbase",
"phoenix.zookeeper.client.port" = "2181",
"phoenix.rowkeys" = "PART_ID",
"phoenix.column.mapping" = "part_id:PART_ID,sourcedb_commit_timestamp:COMMIT_TIMESTAMP"
);

Query :

hive> select * from TEST_HIVE2 where sourcedb_commit_timestamp between '2018-03-01 01:00:00.000' and  '2018-03-20 01:00:00.000';
OK
Failed with exception java.io.IOException:java.lang.RuntimeException: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. TIMESTAMP and VARCHAR for "sourcedb_commit_timestamp" >= '2018-03-01 01:00:00.000'

That happens because we don't use mapped column name when we check whether we need to apply to_timestamp/to_date function. For the default mapping, we regexp patterns don't take into account that column name is double quoted.