[NIFI-3958] Avro decimal logical type fail if database returns Zero precision

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.0
• Component/s: Extensions
• Assignee: Koji Kawamura
• Reporter: Koji Kawamura
• Created: 23/May/17 09:14
• Updated: 23/Nov/20 14:39
• Resolved: 25/May/17 18:41

Description

NIFI-2624 introduced Avro logical type mapping capability. For NUMERIC/DECIMAL types, 'decimal' logical type is used, and positive precision is required to encode a numeric value as 'decimal logical type.

However, database can return 0 (Zero) precision to represent unlimited precision, if a column is not configured with specific precision (some database engine has default precision, but some don't), or numeric value generated by functions or other means dynamically.

We need to provide a way to define default precision in case Zero precision is returned, probably by adding another Processor property to specify default precision.

2017-05-23 16:54:12,414 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: Invalid decimal precision: 0 (must be positive)
        at org.apache.avro.LogicalTypes$Decimal.validate(LogicalTypes.java:206)
        at org.apache.avro.LogicalType.addToSchema(LogicalType.java:70)
        at org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes.java:182)
        at org.apache.nifi.processors.standard.util.JdbcCommon.lambda$createSchema$0(JdbcCommon.java:463)
        at org.apache.nifi.processors.standard.util.JdbcCommon.addNullableField(JdbcCommon.java:359)
        at org.apache.nifi.processors.standard.util.JdbcCommon.createSchema(JdbcCommon.java:461)
        at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:192)
        at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:204)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
        at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:195)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:144)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)

Issue Links

• relates to: NIFI-7973 Oracle JDBC - ExecuteSqlRecord and other components fail on zero precision
• links to: GitHub Pull Request #1851