Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5316

Potential bug with Fetch Parquet

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.8.0, 1.7.1
    • Component/s: None
    • Labels:
      None

      Description

      I have used PutParquet to create Parquet files from Nifi Records.  In trying to convert the parquet back to json by using FetchParquet, I keep getting errors on what seems like any array field.  I have had successes on data that contained no arrays.

      The error in the log is cannont convert a value [xxx, yyy] where xxx and yyy are random strings of type class java.util.ArrayList because no compatible type exist in the UNION of field z.

      I am also see the same error when the value is [] (empty array) as well.  

      I have been successful gong from json to avro and avro to json with the same schema registry values.

      I am defining array fields as follows:

      "type" : [

        "null",

       

      {       "type" : "array",        "items" : "string" }

      ],

      "default": null

      }

       

       

        Attachments

          Activity

            People

            • Assignee:
              bbende Bryan Bende
              Reporter:
              rbruno Robert Bruno

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment