Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4383

Enable block size support in ParquetIO

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-ideas, io-java-parquet
    • None

    Description

      Parquet API allows block size support, which can improve IO performance when working with Parquet files. Currently, the ParquetIO does not support it at all so it looks like a room for improvement for this IO.

      Good intro into this topic: https://www.dremio.com/tuning-parquet/ 

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-10842 Record level progress tracking for BlockTracker

          • P3 - Non-urgent bugs, features, and improvements
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. BEAM-214 Create Parquet IO

          • P3 - Non-urgent bugs, features, and improvements
          • Resolved

          Activity

            People

              Unassigned Unassigned
              ŁukaszG Lukasz Gajowy
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: