[NIFI-4680] Improve error handling in Publish/Consume Kafka processors

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Extensions
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 08/Dec/17 16:16
• Updated: 08/Dec/17 21:13
• Resolved: 08/Dec/17 21:13

Description

While reviewing NIFI-4639, I encountered a couple of issues surrounding how NiFi handles errors in the Publish and Consume Kafka processors.

If interacting with the Hortonworks Schema Registry and unable to connect to it, the Exception that was thrown was a RuntimeException instead of an IOException. This resulted in ConsumeKafkaRecord continuing to try to parse every record it received. This could cause it to take a very long time to stop the processor in such a case.

On the publisher side if this happened, some flowfiles were transferred back to their original queues and then attempted to transfer to failure. As a result, the session would rollback instead of transferring anything to failure, and an error message would indicate that flowfile was already transferred.

When attempting to rollback consumed records, a NPE was thrown if reading from the beginning of the topic (no offsets had been committed).

Issue Links

• links to: GitHub Pull Request #2330