[NIFI-1613] ConvertJSONToSQL Drops Type Information

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.1, 0.5.1
• Fix Version/s: 1.4.0
• Component/s: Core Framework
• Labels: ConvertJSONToSQL Phoenix SQL
• Environment: Ubuntu 14.04 LTS
• Assignee: Toivo Adams
• Reporter: Aaron Stephens
• Created: 09/Mar/16 22:05
• Updated: 13/Jul/17 04:33
• Resolved: 12/Jul/17 21:19

Description

It appears that the ConvertJSONToSQL processor is turning Boolean (and possibly Integer and Float) values into Strings. This is okay for some drivers (like PostgreSQL) which can coerce a String back into a Boolean, but it causes issues for others (specifically Phoenix in my case).

org.apache.phoenix.schema.ConstraintViolationException: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN
        at org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) ~[na:na]
        at org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na]
        at org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) ~[na:na]
        at org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) ~[na:na]
        at org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) ~[na:na]
        at org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) ~[na:na]
        at org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) ~[na:na]
        at org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na]
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-0.4.1.jar:0.4.1]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-0.4.1.jar:0.4.1]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-0.4.1.jar:0.4.1]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-0.4.1.jar:0.4.1]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-0.4.1.jar:0.4.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_79]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_79]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN
        at org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) ~[na:na]
        at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) ~[na:na]
        ... 20 common frames omitted

Issue Links

• relates to: NIFI-2625 ConvertJsonToSql truncates SQL timestamp values
• relates to: PHOENIX-2757 Phoenix Can't Coerce String to Boolean
• links to: GitHub Pull Request #293
• links to: GitHub Pull Request #1976