Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-1356

Create Avro Schema by inferring CSV and JSON data

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.4.1
    • 0.5.0
    • Extensions

    Description

      There are several situations where the ability to dynamically create an Avro schema is desired. Kite provides the ability to dynamically infer an Avro schema from both CSV and JSON data. Since NiFi already contains a kite bundle for converting csv and json to Avro this feature should be an easy add.

      I propose 2 new processors "InferAvroSchemaFromCSV" and "InferAvroSchemaFromJSON". These processors will reside inside of the existing "nifi-kite-bundle" and extend upon the already present third party libraries. Each processor will accept either CSV or JSON and produce as output the Avro Schema JSON and the original data that was presented to them. The processors will rely on kite to perform the actually inferring of the schema.

      Attachments

        Activity

          People

            jeremy.dyer Jeremy Dyer
            jeremy.dyer Jeremy Dyer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified