[FLINK-5826] UDF/UDTF should support variable types and variable arguments

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Assignee: Zhuoluo Yang
• Reporter: Zhuoluo Yang
• Created: 17/Feb/17 07:19
• Updated: 13/Mar/17 15:07
• Resolved: 13/Mar/17 15:07

Description

In some cases, UDF/UDTF should support variable types and variable arguments. Many UDF/UDTF developers wish to make the # of arguments and types flexible to users. They try to make their functions flexible.

Thus, we should support the following styles of UDF/UDTFs.

for example 1, in Java

public class SimpleUDF extends ScalarFunction {
	public int eval(Object... args) {
		// do something
	}
}

for example 2, in Scala

class SimpleUDF extends ScalarFunction {
  def eval(args: Any*): Int = {
    // do something
  }
}

If we modify the code in UserDefinedFunctionUtils.getSignature() and make both signatures pass. The first example will work normally. However, the second example will raise an exception.

Caused by: org.codehaus.commons.compiler.CompileException: Line 58, Column 0: No applicable constructor/method found for actual parameters "java.lang.String"; candidates are: "public java.lang.Object test.SimpleUDF.eval(scala.collection.Seq)"
  at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11523) ~[janino-3.0.6.jar:?]
  at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:8679) ~[janino-3.0.6.jar:?]
  at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:8539) ~[janino-3.0.6.jar:?]

The reason is that Scala will do a sugary modification to the signature of the method. The mothod

 def eval(args: Any*)

will become

def eval(args: scala.collection.Seq<Any>)

in the class file.

The code generation has been done in Java. If we use java style

eval(Object... args)

to call the Scala method, it will raise the above exception.

However, I can't always restrict users to use Java to write a UDF/UDTF. Any ideas in variable types and variable arguments of Scala UDF/UDTFs to prevent the compilation failure?

Sub-Tasks

• ScalarFunction(UDF) should support variable types and variable arguments: Resolved
• TableFunction (UDTF) should support variable types and variable arguments: Resolved