[PARQUET-739] Rle-decoding uses static buffer that is shared accross threads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: cpp-1.0.0
• Component/s: parquet-cpp
• Assignee: Deepak Majeti
• Reporter: Florian Scheibner
• Created: 06/Oct/16 00:00
• Updated: 11/Oct/16 02:48
• Resolved: 11/Oct/16 02:48

Description

Reading two parquet files in parallel lead to a memory corruption that caused a crash. The columns are rle dictionary encoded strings in an uncompressed page, created with parquet-mr.

Initial debugging showed that the indices for the dictionary returned by the rle decoder are garbage. So that data page got corrupted in memory. Reading the files in one thread works.

I have a ColumnReader for each column and read one element from reach column to get a complete row.

The indices are decoded into one global static buffer. So multiple threads all use the same buffer and overwrite each other's indices.