Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7502

Incorrect/invalid codegen for typeof() with UNION

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.17.0
    • Fix Version/s: 1.18.0
    • Component/s: None
    • Labels:

      Description

      The typeof() function is defined as follows:

        @FunctionTemplate(names = {"typeOf"},
                scope = FunctionTemplate.FunctionScope.SIMPLE,
                nulls = NullHandling.INTERNAL)
        public static class GetType implements DrillSimpleFunc {
      
          @Param
          FieldReader input;
          @Output
          VarCharHolder out;
          @Inject
          DrillBuf buf;
      
          @Override
          public void setup() {}
      
          @Override
          public void eval() {
            String typeName = input.getTypeString();
            byte[] type = typeName.getBytes();
            buf = buf.reallocIfNeeded(type.length);
            buf.setBytes(0, type);
            out.buffer = buf;
            out.start = 0;
            out.end = type.length;
          }
        }
      

      Note that the input field is defined as FieldReader which has a method called getTypeString(). As a result, the code works fine in all existing tests in TestTypeFns.

      I tried to add a function to use typeof() on a column of type UNION. When I did, the query failed with a compile error in generated code:

      SYSTEM ERROR: CompileException: Line 42, Column 43: 
        A method named "getTypeString" is not declared in any enclosing class nor any supertype, nor through a static import
      

      The stack trace shows the generated code; Note that the type of input changes from a reader to a holder, causing code to be invalid:

      public class ProjectorGen0 {
      
          DrillBuf work0;
          UnionVector vv1;
          VarCharVector vv6;
          DrillBuf work9;
          VarCharVector vv11;
          DrillBuf work14;
          VarCharVector vv16;
      
          public void doEval(int inIndex, int outIndex)
              throws SchemaChangeException
          {
              {
                  UnionHolder out4 = new UnionHolder();
                  {
                      out4 .isSet = vv1 .getAccessor().isSet((inIndex));
                      if (out4 .isSet == 1) {
                          vv1 .getAccessor().get((inIndex), out4);
                      }
                  }
                  //---- start of eval portion of typeOf function. ----//
                  VarCharHolder out5 = new VarCharHolder();
                  {
                      final VarCharHolder out = new VarCharHolder();
                      UnionHolder input = out4;
                      DrillBuf buf = work0;
                      UnionFunctions$GetType_eval:
      {
          String typeName = input.getTypeString();
          byte[] type = typeName.getBytes();
      
          buf = buf.reallocIfNeeded(type.length);
          buf.setBytes(0, type);
          out.buffer = buf;
          out.start = 0;
          out.end = type.length;
      }
      

      By contrast, here is the generated code for one of the existing TestTypeFns tests where things work:

      public class ProjectorGen0
          extends ProjectorTemplate
      {
      
          DrillBuf work0;
          NullableBigIntVector vv1;
          VarCharVector vv7;
      
          public ProjectorGen0() {
              try {
                  __DRILL_INIT__();
              } catch (SchemaChangeException e) {
                  throw new UnsupportedOperationException(e);
              }
          }
      
          public void doEval(int inIndex, int outIndex)
              throws SchemaChangeException
          {
              {
                 ..
                  //---- start of eval portion of typeOf function. ----//
                  VarCharHolder out6 = new VarCharHolder();
                  {
                      final VarCharHolder out = new VarCharHolder();
                      FieldReader input = new NullableIntHolderReaderImpl(out5);
                      DrillBuf buf = work0;
                      UnionFunctions$GetType_eval:
      {
          String typeName = input.getTypeString();
          byte[] type = typeName.getBytes();
      
          buf = buf.reallocIfNeeded(type.length);
          buf.setBytes(0, type);
          out.buffer = buf;
          out.start = 0;
          out.end = type.length;
      }
                      work0 = buf;
                      out6 .start = out.start;
                      out6 .end = out.end;
                      out6 .buffer = out.buffer;
                  }
                  //---- end of eval portion of typeOf function. ----//
      

      Notice that the input variable is of type FieldReader as expected.

      Queries that work:

          String sql = "SELECT typeof(CAST(a AS " + castType + ")) FROM (VALUES (1)) AS T(a)";
          sql = "SELECT typeof(CAST(a AS " + castType + ")) FROM cp.`functions/null.json`";
          String sql = "SELECT typeof(" + expr + ") FROM (VALUES (" + value + ")) AS T(a)";
      

      Query that fails:

          String sql ="SELECT typeof(a) AS t, modeof(a) as m, drilltypeof(a) AS dt\n" +
                      "FROM cp.`jsoninput/union/c.json`";
      

      The queries that work all include either a CAST or constant values. The query that fails works with data read from a file. Also, the queries that work use scalar types, the query that fails uses the UNION type.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Paul.Rogers Paul Rogers
                Reporter:
                Paul.Rogers Paul Rogers
                Reviewer:
                Vova Vysotskyi
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: