[NIFI-5706] Processor ConvertAvroToParquet

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.1
• Fix Version/s: 1.9.0
• Component/s: Extensions
• Labels: pull-request-available
• Assignee: Bryan Bende
• Reporter: Mohit
• Created: 16/Oct/18 06:34
• Updated: 23/Oct/18 18:09
• Resolved: 23/Oct/18 18:09

Description

Why?

PutParquet support is limited to HDFS.
PutParquet bypasses the flowfile implementation and writes the file directly to sink.
We need a processor for parquet that works like ConvertAvroToOrc.

What?

ConvertAvroToParquet will convert the incoming avro flowfile to a parquet flowfile. Unlike PutParquet, which writes to the hdfs file system, processor ConvertAvroToParquet would write into the flowfile, which can be pipelined to put into other sinks, like local, S3, Azure data lake etc.

 

Issue Links

• links to: GitHub Pull Request #3079