[DRILL-6927] Query fails when hive table with timestamp data is queried with enabled int96_as_timestamp and optimize_scan_with_native_reader options

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.16.0
• Assignee: Vova Vysotskyi
• Reporter: Vova Vysotskyi
• Reviewer: Vitalii Diravka
• Created: 24/Dec/18 16:22
• Updated: 02/Mar/19 03:58
• Resolved: 02/Mar/19 03:58

Description

Steps to reproduce:
1. Create hive table with timestamp column:

create table test_timestamp stored as PARQUET as select timestamp '2018-01-01 12:12:12.123' as c1;

2. Enable store.parquet.reader.int96_as_timestamp and store.hive.parquet.optimize_scan_with_native_reader:

set `store.parquet.reader.int96_as_timestamp`=true;
set `store.hive.parquet.optimize_scan_with_native_reader`=true;

3. Query hive table using Drill:

select * from hive.test_timestamp;

Query fails with error:

Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize incoming schema.  Errors:
 
Error in expression at index -1.  Error: Missing function implementation: [convert_fromtimestamp_impala(TIMESTAMP-OPTIONAL)].  Full expression: --UNKNOWN EXPRESSION--..

Fragment 0:0

Stack trace:

Error in expression at index -1.  Error: Missing function implementation: [convert_fromtimestamp_impala(TIMESTAMP-OPTIONAL)].  Full expression: --UNKNOWN EXPRESSION--..
	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:498) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:583) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:101) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:143) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:297) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:284) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_141]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_141]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) ~[hadoop-common-2.7.0-mapr-1808.jar:na]
	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:284) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
	... 4 common frames omitted

Issue Links

• links to: GitHub Pull Request #1655