[IMPALA-3552] Make incremental stats max serialized size configurable or dramatically increase limit.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.8.0
• Component/s: Catalog
• Labels: usability
• Target Version: Impala 2.8.0
• Assignee: Bharath Vissapragada
• Reporter: Alexander Behm
• Created: 17/May/16 07:32
• Updated: 14/Mar/18 23:43
• Resolved: 15/Nov/16 04:13

Description

The fix for IMPALA-2648/IMPALA-2664 introduced a conservative limitation on the maximum serialized size of incremental stats. As a result, some users with very large tables experienced regressions when upgrading because incremental stats did not work anymore.

We should consider making the limit configurable, or even better fix the design to allow a much higher limit.

Dimitris, assigning to you for triage/consideration.

Issue Links

• is related to: IMPALA-3494 Thrift buffer overflows when serialize more than 3355443200 bytes in impala