[NIFI-2811] JdbcCommon doesn't convert bigint(20) column.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.1.0
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Byunghwa Yun
• Created: 23/Sep/16 06:32
• Updated: 27/Sep/16 14:19
• Resolved: 27/Sep/16 14:19

Description

JdbcCommon doesn't convert SQL-to-Avro that has bigint(20) column with the small value.
Even column type is BIGINT(20), MySQL JDBC driver can return number type value when the return value is small.

Related Exception:
2016-09-23 10:50:22,191 ERROR [Timer-Driven Process Thread-8] o.a.n.p.standard.QueryDatabaseTable
org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records to Avro.
at org.apache.nifi.processors.standard.QueryDatabaseTable.lambda$onTrigger$0(QueryDatabaseTable.java:255) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.QueryDatabaseTable$$Lambda$132/464322944.process(Unknown Source) ~[na:na]
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2123) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:249) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_51]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","string"]: 936398
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:296) ~[avro-1.7.7.jar:1.7.7]
at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:207) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processors.standard.QueryDatabaseTable.lambda$onTrigger$0(QueryDatabaseTable.java:253) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
... 14 common frames omitted
Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["null","string"]: 936398
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58) ~[avro-1.7.7.jar:1.7.7]
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:290) ~[avro-1.7.7.jar:1.7.7]
... 16 common frames omitted

Issue Links

• is related to: NIFI-2531 SQL-to-Avro processors do not convert BIGINT correctly
• links to: GitHub Pull Request #1053