[IMPALA-4795] TCatalogObjectFromObjectName is broken for functions

• Type: Bug
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.10.0
• Component/s: Catalog
• Labels: ramp-up
• Target Version: Product Backlog
• Assignee: Bikramjeet Vig
• Reporter: Henry Robinson
• Created: 19/Jan/17 21:08
• Updated: 05/Apr/18 18:27

Description

TCatalogObjectFromObjectName in catalog-util.cc doesn't set a couple of important fields properly. Neither arg_types nor ret_type are set. Calling TCatalogObjectFromObjectName() for a function appears to always throw an NPE in Java:

I0119 10:57:39.461402 28333 jni-util.cc:169] java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
	at org.apache.impala.catalog.Type.fromThrift(Type.java:379)
	at org.apache.impala.catalog.Function.fromThrift(Function.java:372)
	at org.apache.impala.catalog.Catalog.getTCatalogObject(Catalog.java:461)
	at org.apache.impala.service.JniCatalog.getCatalogObject(JniCatalog.java:202)

This is because ret_type is not properly set. It doesn't need to be set to the real ret-type for the purpose of looking up the function, but does need to be well formed.

Fixing that leads to the following error:

Error: CatalogException: Function not found: TFunction(name:TFunctionName(db_name:_impala_builtins, function_name:abs), binary_type:BUILTIN, arg_types:[], ret_type:TColumnType(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:INT))]), has_var_args:false, signature:abs(BIGINT))

Note arg_types:[] - this is because they're not set, but Db.getFunction() ultimately relies on them being set to compare the functions.

It's easy to repro - hit this URL on an Impalad:

/catalog_object?object_type=FUNCTION&object_name=_impala_builtins.abs(BIGINT)

You can also see lots of these errors if you run impala-py.test test_udfs.py -k native in the Impalad logs. They don't seem to inhibit correct execution; DROP statements seem to work, but it still seems very buggy.

Issue Links

• depends upon: IMPALA-6215 Race between lib_cache and java udf class loading