[PARQUET-378] Add thoroughly parquet test encodings

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-mr
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 23/Sep/15 15:53
• Updated: 21/Apr/18 12:38
• Resolved: 19/Nov/15 18:47

Description

PARQUET-246 highlighted the need to have more thorough validation of the Parquet encodings. With the 2.0 encodings as high priority, we need to validate each encoding with its spec (and ensure there is a spec).

We also need to add a suite of validations that are used for all encodings that test:

• Writing multiple pages (v1 and v2 pages) and multiple row groups
• Validating that each page can be read individually or with last-to-first page order and has the correct set of values (would have caught PARQUET-246).

Issue Links

• links to: Pull Request 274