[HOP-4499] Table Input - Generic Database - Dremio - Meta for Dates comes back as None

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.2.0
• Fix Version/s: Migrated to GHI
• Assignee: Unassigned
• Reporter: Brandon Jackson
• Created: 24/Oct/22 22:22
• Updated: 20/Mar/23 14:52
• Resolved: 20/Mar/23 14:52

Description

Any Date or Timestamp datatype in Dremio seems to show up as None type in Hop

 

I tested the connections and queries with every other column type and did not encounter this message.

When trying to perform very simple queries that succeed on other Java based SQL clients, an error is thrown in Hop like the following while previewing or running a pipeline.  Exploring the database, and previewing first 100 rows and those things from the database setup panel all seem to work okay.

Example Query:

select BATCH_RUN_DT
from Empower.datastaging.stg_comprehensive_log_of_prescriptions
GROUP BY BATCH_RUN_DT

 

2022/10/24 17:08:08 - Hop - Projects enabled
2022/10/24 17:08:08 - Hop - Enabling project : 'empower-pharmacy'
2022/10/24 17:08:09 - GUI - Found git project for: /opt/empower-etl
2022/10/24 17:08:22 - Preview pipeline for Dremio BATCH_RUN_DT - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/10/24 17:08:22 - Preview pipeline for Dremio BATCH_RUN_DT - Execution started for pipeline [Preview pipeline for Dremio BATCH_RUN_DT]
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - ERROR: Unexpected error
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - ERROR: org.apache.hop.core.exception.HopDatabaseException: 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Couldn't get row from result set
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Unable to get value 'None' from database resultset, index 0
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - ResultSet.wasNull() called without a preceding call to a column getter method since the last call to ResultSet.next()
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.Database.getRow(Database.java:2410)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.Database.getRow(Database.java:2380)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.Database.getRow(Database.java:2358)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.pipeline.transforms.tableinput.TableInput.doQuery(TableInput.java:264)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.pipeline.transforms.tableinput.TableInput.processRow(TableInput.java:142)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:51)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at java.base/java.lang.Thread.run(Thread.java:829)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Caused by: org.apache.hop.core.exception.HopDatabaseException: 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Unable to get value 'None' from database resultset, index 0
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - ResultSet.wasNull() called without a preceding call to a column getter method since the last call to ResultSet.next()
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - 
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.row.value.ValueMetaBase.getValueFromResultSet(ValueMetaBase.java:5480)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.BaseDatabaseMeta.getValueFromResultSet(BaseDatabaseMeta.java:1751)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.DatabaseMeta.getValueFromResultSet(DatabaseMeta.java:2170)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.database.Database.getRow(Database.java:2402)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     ... 6 more
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Caused by: com.dremio.jdbc.JdbcApiSqlException: ResultSet.wasNull() called without a preceding call to a column getter method since the last call to ResultSet.next()
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at com.dremio.jdbc.impl.DremioAccessorList.wasNull(DremioAccessorList.java:86)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at com.dremio.jdbc.impl.DremioCursor.wasNull(DremioCursor.java:682)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at cdjd.org.apache.calcite.avatica.AvaticaResultSet.wasNull(AvaticaResultSet.java:235)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at com.dremio.jdbc.impl.DremioResultSetImpl.wasNull(DremioResultSetImpl.java:158)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     at org.apache.hop.core.row.value.ValueMetaBase.getValueFromResultSet(ValueMetaBase.java:5474)
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 -     ... 9 more
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Finished reading query, closing connection.
2022/10/24 17:08:23 - Dremio BATCH_RUN_DT.0 - Finished processing (I=1, O=0, R=0, W=0, U=0, E=1)
2022/10/24 17:08:23 - Preview pipeline for Dremio BATCH_RUN_DT - Pipeline duration : 0.994 seconds [  0.994" ]
2022/10/24 17:08:23 - Preview pipeline for Dremio BATCH_RUN_DT - Execution finished on a local pipeline engine with run configuration 'local'
2022/10/24 17:08:23 - Preview pipeline for Dremio BATCH_RUN_DT - Pipeline detected one or more transforms with errors.
2022/10/24 17:08:23 - Preview pipeline for Dremio BATCH_RUN_DT - Pipeline is killing the other transforms!