[PARQUET-324] row count incorrect if data file has more than 2^31 rows

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.7.0, 1.8.0
• Fix Version/s: 1.8.0
• Component/s: parquet-mr
• Assignee: Thomas Friedrich
• Reporter: Thomas Friedrich
• Created: 01/Jul/15 20:25
• Updated: 23/Jun/24 03:27
• Resolved: 03/Jul/15 17:54

Description

If a parquet file has more than 2^31 rows, the row count written into the file metadata is incorrect.
The cause of the problem is the use of an int instead of long data type for numRows in ParquetMetadataConverter, toParquetMetadata:
int numRows = 0;
for (BlockMetaData block : blocks)

{ numRows += block.getRowCount(); addRowGroup(parquetMetadata, rowGroups, block); }

Issue Links

• links to: PR #233