[PARQUET-1537] [C++] The patch for PARQUET-1508 leads to infinite loop and infinite memory allocation when reading very sparse ByteArray columns

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: cpp-4.0.0
• Component/s: parquet-cpp
• Assignee: Hatem Helal
• Reporter: Valery Meleshkin
• Created: 21/Feb/19 15:56
• Updated: 21/Mar/19 10:07
• Resolved: 21/Mar/19 10:07

Description

In this loop

https://github.com/apache/arrow/commit/3d435e4f8d5fb7a54a4a9d285e1a42d60186d8dc#diff-47fe879cb9baad6c633c55f0a34a09c3R739

The branch of if dealing with null values does not increment variable 'i'. Therefore on chunks containing only NULLs once a thread enters the loop, it stays in that loop forever. I'm not entirely sure whether 'num_values' variable was meant to be the number of non-NULL values, yet the total number of values is passed here https://github.com/apache/arrow/blob/3d435e4f8d5fb7a54a4a9d285e1a42d60186d8dc/cpp/src/parquet/arrow/record_reader.cc#L528

 

On my local machine adding `++i` to the NULL-handling branch seems to fix the problem. Unfortunately, I'm not familiar with the codebase enough to be certain it's a proper fix.