Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-869

Min/Max record counts for block size checks are not configurable

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      While the min/max record counts for page size check are configurable via ParquetOutputFormat.MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK and ParquetOutputFormat.MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK configs and via ParquetProperties directly, the min/max record counts for block size check are hard coded inside InternalParquetRecordWriter.

      These two settings should also be configurable.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-409 InternalParquetRecordWriter doesn't use min/max row counts

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #470

          Web Link PR #447 (replaced by #470)

          Web Link PR #470

          Activity

            People

              Unassigned Unassigned
              pradeepg26 Pradeep Gollakota
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: