Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8439

Handle parquet INT96 timestamps as byte-arrays (instead of exception)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.14.0
    • None
    • None

    Description

      Currently trying to read parquet files with INT96 timestamps results in an exception.
      Updating parquet-avro to 1.12.0 allows to read them as byte arrays. With some changes in NiFi we can also write them as such back to parquet.
      Not perfect but still better than throwing an exception.

      Attachments

        Issue Links

          fixes

          Bug - A problem which impairs or prevents the functions of the product. NIFI-8292 ParquetReader can't read FlowFile, which was written by ParquerRecordSetWriter

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. NIFI-8436 update the apache parquet library version (to enable zstd compression)

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #5006

          Activity

            People

              tpalfy Tamas Palfy
              tpalfy Tamas Palfy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h