Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4795

TCatalogObjectFromObjectName is broken for functions

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Catalog
    • Labels:

      Description

      TCatalogObjectFromObjectName in catalog-util.cc doesn't set a couple of important fields properly. Neither arg_types nor ret_type are set. Calling TCatalogObjectFromObjectName() for a function appears to always throw an NPE in Java:

      I0119 10:57:39.461402 28333 jni-util.cc:169] java.lang.IllegalStateException
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
      	at org.apache.impala.catalog.Type.fromThrift(Type.java:379)
      	at org.apache.impala.catalog.Function.fromThrift(Function.java:372)
      	at org.apache.impala.catalog.Catalog.getTCatalogObject(Catalog.java:461)
      	at org.apache.impala.service.JniCatalog.getCatalogObject(JniCatalog.java:202)
      

      This is because ret_type is not properly set. It doesn't need to be set to the real ret-type for the purpose of looking up the function, but does need to be well formed.

      Fixing that leads to the following error:

      Error: CatalogException: Function not found: TFunction(name:TFunctionName(db_name:_impala_builtins, function_name:abs), binary_type:BUILTIN, arg_types:[], ret_type:TColumnType(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:INT))]), has_var_args:false, signature:abs(BIGINT))
      

      Note arg_types:[] - this is because they're not set, but Db.getFunction() ultimately relies on them being set to compare the functions.

      It's easy to repro - hit this URL on an Impalad:

      /catalog_object?object_type=FUNCTION&object_name=_impala_builtins.abs(BIGINT)

      You can also see lots of these errors if you run impala-py.test test_udfs.py -k native in the Impalad logs. They don't seem to inhibit correct execution; DROP statements seem to work, but it still seems very buggy.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bikramjeet.vig Bikramjeet Vig
                Reporter:
                henryr Henry Robinson
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: