[IMPALA-3820] Handle linkage errors gracefully while loading java udfs from Hive

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.6.2
• Component/s: Catalog
• Labels: crash downgraded
• Target Version: Impala 2.7.0, Impala 2.6.2, Impala 2.5.4
• Assignee: Bharath Vissapragada
• Reporter: Bharath Vissapragada
• Created: 05/Jul/16 08:43
• Updated: 08/Sep/16 16:37
• Resolved: 25/Aug/16 00:59

Description

Currently we don't handle "Error"s [0] while trying to load java udfs from Hive. Although we catch the generic "Exception" in the code, sometimes class loaders throw "Error"s. which are fatal for catalog jvm and aborts the Catalog startup.

    try {
     /* Loading logic */
    } catch (ClassNotFoundException c) {
      String errorMsg = "Error loading Java function: " + db + "." +
          function.getFunctionName() + ". Symbol class " + udfClass +
          "not found in Jar: " + jarUri;
      LOG.error(errorMsg);
      throw new ImpalaRuntimeException(errorMsg, c);
    } catch (Exception e) {
      LOG.error("Skipping function load: " + function.getFunctionName(), e);
      throw new ImpalaRuntimeException("Error extracting functions", e);
    }

Generally Java doesn't encourage to catch errors (for ex: OOM). However while loading 3rd party jars during runtime, class loaders can throw LinkageError [1,2], which the catalog is expected to handle. For example, a jar might be present while compiling the udf, but might be missing during runtime, which throws NoClassDefError. These errors are generally a subset of LinkageErrors, which the catalog must gracefully handle.

Workaround: Drop/fix the offending functions from hive and restart catalog

[0] https://docs.oracle.com/javase/7/docs/api/java/lang/Error.html
[1] https://docs.oracle.com/javase/7/docs/api/java/lang/LinkageError.html
[2] http://stackoverflow.com/questions/352780/when-to-catch-java-lang-error