[HIVE-22476] Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Slim Bouguerra
• Reporter: Slim Bouguerra
• Created: 09/Nov/19 00:16
• Updated: 27/Feb/24 22:23
• Resolved: 02/Jun/20 03:29

Description

The actual issue stems to the different date parser used by various part of the engine.
Fetch task uses udfdatediff via

 org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate

while the vectorized llap execution uses

VectorUDFDateDiffScalarCol

.

This fix is meant to be not very intrusive and will add more support to the GenericUDFToDate by enhancing the parser.
For the longer term will be better to use one parser for all the operators.

Thanks Rajkumar Singh for the repro example

 
create external table testdatediff(datetimecol string) stored as orc;
insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
select datetimecol from testdatediff where datediff(cast(current_timestamp as string), datetimecol)<183;

set hive.ferch.task.conversion=none;
select datetimecol from testdatediff where datediff(cast(current_timestamp as string), datetimecol)<183;

Issue Links

• is duplicated by: HIVE-22459 Hive datadiff function provided inconsistent results when hive.ferch.task.conversion is set to none
• links to: GitHub Pull Request #1046