[PARQUET-2124] Bad DCHECK For Intermixed Dictionary Encoding

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: cpp-8.0.0
• Component/s: parquet-cpp
• Labels: pull-request-available
• Assignee: William Butler
• Reporter: William Butler
• Created: 14/Feb/22 23:57
• Updated: 04/Mar/22 23:54
• Resolved: 15/Feb/22 17:45

Description

Parquet CPP has a DCHECK for a dictionary encoded page coming after a non-dictionary encoded page. This is bad because the DCHECK can be triggered by Parquet files that have a column that has a dictionary page, then a non-dictionary encoded page, then a page of dictionary encoded values(indices). Fuzzing found such a file. While this could be turned into an exception, I don't see anything in the Parquet specification that prohibits such an occurrence of pages.

This situation has brought up on the mailing list before(https://lists.apache.org/thread/3bzymmbxvmzj12km7cjz1150ndvy9bos) and it seems like this is valid but nobody is doing it.

In the PR that added this check(https://github.com/apache/parquet-cpp/pull/73) it was noted that the check is probably not needed.

Issue Links

• links to: GitHub Pull Request #76
• links to: GitHub Pull Request #12427