[IMPALA-3499] Backend cannot write catalog-update topic more than 2GB to jByteArray

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.0
• Fix Version/s: Impala 2.7.0, Impala 2.5.2, Impala 2.6.1
• Component/s: Catalog
• Environment: jdk 1.7.0_51
• Target Version: Impala 2.3.5, Impala 2.7.0, Impala 2.5.2, Impala 2.6.1
• Assignee: Huaisi Xu
• Reporter: He Tianyi
• Created: 09/May/16 03:52
• Updated: 14/Dec/16 18:15
• Resolved: 17/Jun/16 18:37

Description

I've encountered a issue after impalad restarts. Logging shows:

E0509 11:44:08.168478 72547 impala-server.cc:1325] There was an error processing the impalad catalog update. Requesting a full topic update to recover: NegativeArraySizeException: null

This may be related to size of catalog-update topic in statestored exceeding 2GB.
In updateCatalogCache, serialized data passed through JNI are represented as byte[], which cannot exceed that limit.

When I restart catalogd (that is, reset catalog-update topic), everything goes fine.

I'm assuming that metadata larger than 2GB can be a common case in massive scale usage.
Is there any possible approach to fix this?
Thanks.

Issue Links

• is related to: IMPALA-3494 Thrift buffer overflows when serialize more than 3355443200 bytes in impala