[PARQUET-1623] [C++] Invalid memory access with a magic number of records

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: cpp-4.0.0
• Component/s: parquet-cpp
• Labels: pull-request-available
• Assignee: Hatem Helal
• Reporter: Hatem Helal
• Created: 11/Jul/19 15:46
• Updated: 12/Jul/19 08:50
• Resolved: 12/Jul/19 08:50

Description

I've observed a crash due to an invalid memory access when trying to read a parquet file that I created with a single column of double-precision values that occupies a fixed amount of memory.  After some experimentation I found that the following unittest added to arrow-reader-writer-test.cc will fail when run in an ASAN build.

TEST(TestArrowReadWrite, MultiDataPageMagicNumber) {
  const int num_rows = 262144;  // 2^18

  std::shared_ptr<Table> table;
  ASSERT_NO_FATAL_FAILURE(MakeDoubleTable(1, num_rows, 1, &table));

  std::shared_ptr<Table> result;
  ASSERT_NO_FATAL_FAILURE(
      DoSimpleRoundtrip(table, false, table->num_rows(), {}, &result));

  ASSERT_NO_FATAL_FAILURE(::arrow::AssertTablesEqual(*table, *result));
}

 

Issue Links

• links to: GitHub Pull Request #4857