I've encountered a issue after impalad restarts. Logging shows:
E0509 11:44:08.168478 72547 impala-server.cc:1325] There was an error processing the impalad catalog update. Requesting a full topic update to recover: NegativeArraySizeException: null
This may be related to size of catalog-update topic in statestored exceeding 2GB.
In updateCatalogCache, serialized data passed through JNI are represented as byte, which cannot exceed that limit.
When I restart catalogd (that is, reset catalog-update topic), everything goes fine.
I'm assuming that metadata larger than 2GB can be a common case in massive scale usage.
Is there any possible approach to fix this?