Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
Now that the specification with respect to page level checksums has been clarified (see https://jira.apache.org/jira/browse/PARQUET-1539), we can actually implement it. What needs to be done:
- Implement calculating and writing checksums on write path
- Implement checksum validation on read path
- Make writing out checksums and validation on reads optional (default off)
- Implement tests
Attachments
Issue Links
- causes
-
PARQUET-1746 Changed the data order after DataFrame reuse
- Resolved
- links to