[DRILL-7502] Incorrect/invalid codegen for typeof() with UNION

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.17.0
• Fix Version/s: 1.18.0
• Labels: ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Vova Vysotskyi
• Created: 30/Dec/19 08:28
• Updated: 09/Jan/20 16:07
• Resolved: 09/Jan/20 16:07

Description

The typeof() function is defined as follows:

  @FunctionTemplate(names = {"typeOf"},
          scope = FunctionTemplate.FunctionScope.SIMPLE,
          nulls = NullHandling.INTERNAL)
  public static class GetType implements DrillSimpleFunc {

    @Param
    FieldReader input;
    @Output
    VarCharHolder out;
    @Inject
    DrillBuf buf;

    @Override
    public void setup() {}

    @Override
    public void eval() {
      String typeName = input.getTypeString();
      byte[] type = typeName.getBytes();
      buf = buf.reallocIfNeeded(type.length);
      buf.setBytes(0, type);
      out.buffer = buf;
      out.start = 0;
      out.end = type.length;
    }
  }

Note that the input field is defined as FieldReader which has a method called getTypeString(). As a result, the code works fine in all existing tests in TestTypeFns.

I tried to add a function to use typeof() on a column of type UNION. When I did, the query failed with a compile error in generated code:

SYSTEM ERROR: CompileException: Line 42, Column 43: 
  A method named "getTypeString" is not declared in any enclosing class nor any supertype, nor through a static import

The stack trace shows the generated code; Note that the type of input changes from a reader to a holder, causing code to be invalid:

public class ProjectorGen0 {

    DrillBuf work0;
    UnionVector vv1;
    VarCharVector vv6;
    DrillBuf work9;
    VarCharVector vv11;
    DrillBuf work14;
    VarCharVector vv16;

    public void doEval(int inIndex, int outIndex)
        throws SchemaChangeException
    {
        {
            UnionHolder out4 = new UnionHolder();
            {
                out4 .isSet = vv1 .getAccessor().isSet((inIndex));
                if (out4 .isSet == 1) {
                    vv1 .getAccessor().get((inIndex), out4);
                }
            }
            //---- start of eval portion of typeOf function. ----//
            VarCharHolder out5 = new VarCharHolder();
            {
                final VarCharHolder out = new VarCharHolder();
                UnionHolder input = out4;
                DrillBuf buf = work0;
                UnionFunctions$GetType_eval:
{
    String typeName = input.getTypeString();
    byte[] type = typeName.getBytes();

    buf = buf.reallocIfNeeded(type.length);
    buf.setBytes(0, type);
    out.buffer = buf;
    out.start = 0;
    out.end = type.length;
}

By contrast, here is the generated code for one of the existing TestTypeFns tests where things work:

public class ProjectorGen0
    extends ProjectorTemplate
{

    DrillBuf work0;
    NullableBigIntVector vv1;
    VarCharVector vv7;

    public ProjectorGen0() {
        try {
            __DRILL_INIT__();
        } catch (SchemaChangeException e) {
            throw new UnsupportedOperationException(e);
        }
    }

    public void doEval(int inIndex, int outIndex)
        throws SchemaChangeException
    {
        {
           ..
            //---- start of eval portion of typeOf function. ----//
            VarCharHolder out6 = new VarCharHolder();
            {
                final VarCharHolder out = new VarCharHolder();
                FieldReader input = new NullableIntHolderReaderImpl(out5);
                DrillBuf buf = work0;
                UnionFunctions$GetType_eval:
{
    String typeName = input.getTypeString();
    byte[] type = typeName.getBytes();

    buf = buf.reallocIfNeeded(type.length);
    buf.setBytes(0, type);
    out.buffer = buf;
    out.start = 0;
    out.end = type.length;
}
                work0 = buf;
                out6 .start = out.start;
                out6 .end = out.end;
                out6 .buffer = out.buffer;
            }
            //---- end of eval portion of typeOf function. ----//

Notice that the input variable is of type FieldReader as expected.

Queries that work:

    String sql = "SELECT typeof(CAST(a AS " + castType + ")) FROM (VALUES (1)) AS T(a)";
    sql = "SELECT typeof(CAST(a AS " + castType + ")) FROM cp.`functions/null.json`";
    String sql = "SELECT typeof(" + expr + ") FROM (VALUES (" + value + ")) AS T(a)";

Query that fails:

    String sql ="SELECT typeof(a) AS t, modeof(a) as m, drilltypeof(a) AS dt\n" +
                "FROM cp.`jsoninput/union/c.json`";

The queries that work all include either a CAST or constant values. The query that fails works with data read from a file. Also, the queries that work use scalar types, the query that fails uses the UNION type.

Issue Links

• incorporates: DRILL-7510 Incorrect String/number comparison with union types
• incorporates: DRILL-6362 typeof() lies about types
• incorporates: DRILL-6360 Document the typeof() function
• is related to: DRILL-5189 There's no documentation for the typeof() function
• links to: GitHub Pull Request #1945