[NIFI-11269] Adding snappy support for SplitAvro

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 2.0.0-M1, 1.21.0
• Component/s: Extensions
• Flags: Patch, Important
• Assignee: john-alan
• Reporter: john-alan
• Created: 12/Mar/23 15:33
• Updated: 16/Mar/23 13:55
• Resolved: 16/Mar/23 13:55

Description

I'm using nifi to split a very big avro file (~500MB), but the processor is throwing this error:

SplitAvro[id=d5f27b12-0186-1000-89b6-3171f53ce832] Processing halted: yielding [1 sec]: org.apache.avro.AvroRuntimeException: Unrecognized codec: snappy

I'm using NiFi 1.20.0
The thing is that in 1.13.2 it worked fine. After upgrading to 1.20.0 the problem accrued.
 
After searching for a bit I've found this solution:
Add to nifi/nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/pom.xml this dependency:

<dependency>
            <groupId>org.xerial.snappy</groupId>
            <artifactId>snappy-java</artifactId>
            <version>1.1.8.4</version>
</dependency>

After build and run this actually works! Would be very nice to add this to the next patch.

 

This is a deal breaker because I want to upgrade to 1.20.0 from 1.13.2 and can't because of this error.

Issue Links

• links to: GitHub Pull Request #7035