Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4795

TCatalogObjectFromObjectName is broken for functions

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • Impala 2.9.0
    • Impala 2.10.0
    • Catalog

    Description

      TCatalogObjectFromObjectName in catalog-util.cc doesn't set a couple of important fields properly. Neither arg_types nor ret_type are set. Calling TCatalogObjectFromObjectName() for a function appears to always throw an NPE in Java:

      I0119 10:57:39.461402 28333 jni-util.cc:169] java.lang.IllegalStateException
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
      	at org.apache.impala.catalog.Type.fromThrift(Type.java:379)
      	at org.apache.impala.catalog.Function.fromThrift(Function.java:372)
      	at org.apache.impala.catalog.Catalog.getTCatalogObject(Catalog.java:461)
      	at org.apache.impala.service.JniCatalog.getCatalogObject(JniCatalog.java:202)
      

      This is because ret_type is not properly set. It doesn't need to be set to the real ret-type for the purpose of looking up the function, but does need to be well formed.

      Fixing that leads to the following error:

      Error: CatalogException: Function not found: TFunction(name:TFunctionName(db_name:_impala_builtins, function_name:abs), binary_type:BUILTIN, arg_types:[], ret_type:TColumnType(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:INT))]), has_var_args:false, signature:abs(BIGINT))
      

      Note arg_types:[] - this is because they're not set, but Db.getFunction() ultimately relies on them being set to compare the functions.

      It's easy to repro - hit this URL on an Impalad:

      /catalog_object?object_type=FUNCTION&object_name=_impala_builtins.abs(BIGINT)

      You can also see lots of these errors if you run impala-py.test test_udfs.py -k native in the Impalad logs. They don't seem to inhibit correct execution; DROP statements seem to work, but it still seems very buggy.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bikramjeet.vig Bikramjeet Vig
            henryr Henry Robinson

            Dates

              Created:
              Updated:

              Slack

                Issue deployment