[DRILL-6167] Table functions give error without hidden type field

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.12.0
• Assignee: Unassigned
• Reporter: Paul Rogers
• Created: 19/Feb/18 06:18
• Updated: 21/Jan/19 13:43

Description

Drill provides table functions (see Using the Formats Attributes as Table Function Parameters) which allow queries to specify properties of format plugins.

All table functions derive from the FormatPluginConfig base class:

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, 
      include = JsonTypeInfo.As.PROPERTY,
      property="type")
public interface FormatPluginConfig { }

The annotation above appears to define a property called type to identify the subtype of the base class, and is used when deserializing JSON for the config object.

Suppose we define a "regex plugin" to let us read a log file using a regex. We define a plugin config for this plugin:

@JsonTypeName("regex")
@JsonInclude(Include.NON_DEFAULT)
public class RegexFormatConfig implements FormatPluginConfig {

  public String regex;
...

For the above, everything works just fine if we use the config in the normal way (define in the Drill web console or programmatically in a test.)

Suppose we want to change the regex in a table function:

SELECT * FROM table(cp.`regex/simple.log2`
  (regex => 'some pattern'))

When run (in the debugger, as a test), we get the following error:

org.apache.calcite.runtime.CalciteContextException: From line 1,
    column 24 to line 2, column 40:
    DEFAULT is only allowed for optional parameters

The error is thrown in SqlOperator.checkOperandTypes() which calls FamilyOperandTypeChecker.isOptional() which calls WorkspaceSchemaFactory.WithOptionsTableMacro.getParameters() which has somehow decided that the type "parameter" is required.

OK, if it is required, let's provide it:

SELECT * FROM table(cp.`regex/simple.log2`
  (type => 'regex', regex => 'some pattern'))

The above SQL works as expected, producing the proper output.

Since the type is hidden, and is known only to Java developers, the code should not require that the user specify it; especially since there is exactly only one correct value.

Issue Links

• Is contained by: DRILL-6986 Table function improvements / issues (UMBRELLA JIRA)
• is related to: DRILL-6168 Table functions do not "inherit" default configuration
• relates to: DRILL-6168 Table functions do not "inherit" default configuration
• links to: GitHub Pull Request #1114