[DRILL-6673] Drill table functions cannot use plugin configs with a constructor

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.13.0
• Assignee: Unassigned
• Reporter: Paul Rogers
• Created: 08/Aug/18 00:30
• Updated: 21/Jan/19 13:43

Description

Consider an example format plugin, such as the regex one used in the Drill book. (GitHub reference needed.) Format configurations should be immutable. So, we make the members private final and define a constructor like this:

public class RegexFormatConfig implements FormatPluginConfig {

  private final String regex;

  // Should be a List<String>. But, table functions don't support
  // lists, so we store the fields as single string that contains
  // a comma-delimited list: a, b, c. Spaces are optional.

  private final String fields;
  private final String extension;

  public RegexFormatConfig(
      @JsonProperty("extension") String extension,
      @JsonProperty("regex") String regex,
      @JsonProperty("fields") String fields
      ) {
    this.regex = regex;
    this.fields = fields;
    this.extension = extension;
  }

We can then create a plugin configuration using the Drill Web console, the bootstrap-storage-plugins.json and so on. All work fine.

Suppose we try to define a configuration using a Drill table function:

      final String sql = "SELECT * FROM table(cp.`regex/simple.log2`\n" +
          "(type => 'regex',\n" +
          " extension => 'log2',\n" +
          " regex => '(\\\\d\\\\d\\\\d\\\\d)-(\\\\d\\\\d)-(\\\\d\\\\d) .*',\n" +
          " fields => 'a, b, c, d'))";

We get this error:

org.apache.drill.common.exceptions.UserRemoteException: PARSE ERROR: 
configuration for format of type regex can not be created 
(class: org.apache.drill.exec.store.easy.regex.RegexFormatConfig)

table regex/simple.log2

They fail because the code that resolves the plugin configuration does not know about the Jackson constructor conventions: it requires an default constructor:

package org.apache.drill.exec.store.dfs;
...
final class FormatPluginOptionsDescriptor {
  ...
  FormatPluginConfig createConfigForTable(TableInstance t) {
    ...
    try {
      config = pluginConfigClass.newInstance();
    } catch (InstantiationException | IllegalAccessException e) {
      throw UserException.parseError(e)
          .message(
              "configuration for format of type %s can not be created (class: %s)",
              this.typeName, pluginConfigClass.getName())
         ...

The workaround would seem to be to define setters using Jackson standards. But, that does not work either due to DRILL-6672 which says that Drill table functions don't allow them. The only real workaround is to make the fields public and omit the constructor.

Issue Links

• Is contained by: DRILL-6986 Table function improvements / issues (UMBRELLA JIRA)