Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3820

Handle linkage errors gracefully while loading java udfs from Hive

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.5.0
    • Fix Version/s: Impala 2.6.2
    • Component/s: Catalog
    • Labels:

      Description

      Currently we don't handle "Error"s [0] while trying to load java udfs from Hive. Although we catch the generic "Exception" in the code, sometimes class loaders throw "Error"s. which are fatal for catalog jvm and aborts the Catalog startup.

          try {
           /* Loading logic */
          } catch (ClassNotFoundException c) {
            String errorMsg = "Error loading Java function: " + db + "." +
                function.getFunctionName() + ". Symbol class " + udfClass +
                "not found in Jar: " + jarUri;
            LOG.error(errorMsg);
            throw new ImpalaRuntimeException(errorMsg, c);
          } catch (Exception e) {
            LOG.error("Skipping function load: " + function.getFunctionName(), e);
            throw new ImpalaRuntimeException("Error extracting functions", e);
          }
      

      Generally Java doesn't encourage to catch errors (for ex: OOM). However while loading 3rd party jars during runtime, class loaders can throw LinkageError [1,2], which the catalog is expected to handle. For example, a jar might be present while compiling the udf, but might be missing during runtime, which throws NoClassDefError. These errors are generally a subset of LinkageErrors, which the catalog must gracefully handle.

      Workaround: Drop/fix the offending functions from hive and restart catalog

      [0] https://docs.oracle.com/javase/7/docs/api/java/lang/Error.html
      [1] https://docs.oracle.com/javase/7/docs/api/java/lang/LinkageError.html
      [2] http://stackoverflow.com/questions/352780/when-to-catch-java-lang-error

        Attachments

          Activity

            People

            • Assignee:
              bharathv bharath v
              Reporter:
              bharathv bharath v
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: