Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-378

Add thoroughly parquet test encodings

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.9.0, 1.8.2
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      PARQUET-246 highlighted the need to have more thorough validation of the Parquet encodings. With the 2.0 encodings as high priority, we need to validate each encoding with its spec (and ensure there is a spec).

      We also need to add a suite of validations that are used for all encodings that test:

      • Writing multiple pages (v1 and v2 pages) and multiple row groups
      • Validating that each page can be read individually or with last-to-first page order and has the correct set of values (would have caught PARQUET-246).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                spena Sergio Peña
                Reporter:
                spena Sergio Peña
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: