[PARQUET-66] InternalParquetRecordWriter int overflow causes unnecessary memory check warning

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.6.0
• Component/s: parquet-mr
• Assignee: Unassigned
• Reporter: Eric Snyder
• Created: 07/Aug/14 13:13
• Updated: 07/Apr/15 20:46
• Resolved: 20/Aug/14 21:10

Description

flushStore() contains a check on the amount of memory allocated and if it is > 3 * blockSize it logs a warning. If blockSize is large enough, e.g. 1GiB then 3 * blockSize will overflow and be negative causing the warning to always be logged.

if (store.allocatedSize() > 3 * blockSize) {
    LOG.warn("Too much memory used: " + store.memUsageString());
}