[IMPALA-8568] Fix Impala JNI warnings when -Xcheck:jni is enabled

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Epic Color: ghx-label-5
• Assignee: Unassigned
• Reporter: Sahil Takiar
• Created: 21/May/19 20:01
• Updated: 14/Aug/23 07:40

Description

IMPALA-8250 made a lot of improvements to our usage of the JNI. Impala no longer crashes when running exhaustive tests with -Xcheck:jni enabled. We made some progress in cleaning up libhdfs JNI usage in HDFS-14321 and HDFS-14348 as well.

However, re-running exhaustive tests with -Xcheck:jni still shows a lot of warnings. It's not clear if these warnings are from libhdfs or Impala, but either way we should drive a fix.

The most concerning of the current list of JNI warnings produced by Impala, are the "JNI call made without checking exceptions when required to from ..." warnings. Essentially, this means that when making a JNI call, we are not properly checking for exceptions. This can be problematic because a JNI call make throw an exception, and we end up swallowing it.

There are lots of warnings about "WARNING: JNI local refs: [x], exceeds capacity: [y]". Based on some digging (e.g. https://community.oracle.com/message/13290783) it looks like these warnings aren't fatal, but are just bad practice. I think we can fix the most egregious offenders (looks like the HBase code is one of them), and hopefully live with the rest (a lot of the warnings are thrown by internal Java code as well).

Issue Links

• is related to: IMPALA-12274 Memory leak because of the local reference created by `NewObject` in class Catalog was not released
• is related to: IMPALA-8250 Impala crashes with -Xcheck:jni