[PARQUET-1702] [C++] Make BufferedRowGroupWriter compatible with parquet encryption

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: cpp-4.0.0
• Fix Version/s: cpp-4.0.0
• Component/s: parquet-cpp
• Labels: pull-request-available
• Assignee: Or Ozeri
• Reporter: Or Ozeri
• Created: 26/Nov/19 08:50
• Updated: 04/Dec/19 17:40
• Resolved: 04/Dec/19 17:40

Description

The newly added parquet encryption feature currently works only with SerializedRowGroupWriter.
There are several issues preventing the use of BufferedRowGroupWriter with encryption enabled:

1. Meta encryptor not passed on to ColumnChunkMetaDataBuilder::Finish. This can trigger a null-pointer dereference (reported as segmentation fault).
2. UpdateEncryption not called on Close, resulting in an incorrect AAD string when encrypting the column chunk metadata.
3. The column ordinal passed on to PageWriter::Open is always zero, resulting in a wrong AAD string when encrypting the columns data (except for the first column).
4. When decrypting a column chunk with no dictionary pages, PARQUET-1706 confuses the decryptor to think it is decrypting a dictionary page, which again causes a wrong AAD string to be used when decrypting.

We propose a patch (few dozen lines) to fix the above issues.
We also extend the current parquet-encryption-test unit test, which tests SerializedRowGroupWriter, to test also with BufferedRowGroupWriter.

Issue Links

• links to: GitHub Pull Request #5903