Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1054

NPE caused by wrong code generation for Timestamp fields

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.6.0, 1.5.0, 1.4.0-incubating
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      Problem occurs when:

      • Execute a query containing 2 checks on a Timestamp field
      • Table contains records which have NULL values for this field

      Example query:

      select * from aTable where aTimestamp > timestamp '2015-1-1 00:00:00' and aTimestamp < timestamp '2015-2-1 00:00:00';
      
      /*  48 */   public boolean moveNext() {
      /*  49 */     while (inputEnumerator.moveNext()) {
      /*  50 */       final java.sql.Timestamp inp23_ = (java.sql.Timestamp) ((Object[]) inputEnumerator.current())[23];
      /*  51 */       final long v = org.apache.calcite.runtime.SqlFunctions.toLong(inp23_);
      /*  52 */       if (inp23_ != null && v > 1420070400000L && (inp23_ != null && v < 1422748800000L)) {
      /*  53 */         return true;
      /*  54 */       }
      /*  55 */     }
      /*  56 */     return false;
      /*  57 */   }
      

      Stack trace snippet

      Caused by: java.lang.NullPointerException
      	at org.apache.calcite.runtime.SqlFunctions.toLong(SqlFunctions.java:1094)
      	at org.apache.calcite.runtime.SqlFunctions.toLong(SqlFunctions.java:1089)
      	at Baz$1$1.moveNext(ANONYMOUS.java:51)
      	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<init>(Linq4j.java:677)
      	at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:103)
      

      The generated code also looks wrong for date fields.

      /*  15 */   public boolean moveNext() {
      /*  16 */     while (inputEnumerator.moveNext()) {
      /*  17 */       final java.sql.Date current = (java.sql.Date) inputEnumerator.current();
      /*  18 */       final int v = org.apache.calcite.runtime.SqlFunctions.toInt(current);
      /*  19 */       if (current != null && v > 2780 && (current != null && v < 5290)) {
      /*  20 */         return true;
      /*  21 */       }
      /*  22 */     }
      /*  23 */     return false;
      /*  24 */   }
      


      Other types of fields do not have this problem. Below is what the generated code looks like in the case of a String field. On line 20 there is a null check. This is the type of check that needs to be generated for Timestamp fields as well.

      select empno from sales.emps where gender > 'A' and gender < 'Z';
      
      /*  17 */  public boolean moveNext() {
      /*  18 */    while (inputEnumerator.moveNext()) {
      /*  19 */      final Object[] current = (Object[]) inputEnumerator.current();
      /*  20 */      final String inp3_ = current[3] == null ? (String) null : current[3].toString();
      /*  21 */      if (inp3_ != null && org.apache.calcite.runtime.SqlFunctions.gt(inp3_, $L4J$C$org_apache_calcite_runtime_SqlFunctions_rtrim_A_) && (inp3_ != null && org.apache.calcite.runtime.SqlFunctions.lt(inp3_, $L4J$C$org_apache_calcite_runtime_SqlFunctions_rtrim_Z_))) {
      /*  22 */        return true;
      /*  23 */      }
      /*  24 */    }
      /*  25 */    return false;
      /*  26 */  }
      

        Issue Links

          Activity

          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          It looks like 1066-oracle branch is using proper unboxed types for aggregate/window functions. In other words, it avoids unbox-add-box overheads. That is good.

          Null semantics for function/operator calls seems to regress a bit (see samples above). It does repeated null checks, and it does some strange things like truncate((String) null, 1)

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - It looks like 1066-oracle branch is using proper unboxed types for aggregate/window functions. In other words, it avoids unbox-add-box overheads. That is good. Null semantics for function/operator calls seems to regress a bit (see samples above). It does repeated null checks, and it does some strange things like truncate((String) null, 1)
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          JdbcTest.testBooleansInWhere:

          /* 252 */               if (((Object[]) inputEnumerator.current())[0] != null && org.apache.calcite.runtime.SqlFunctions.toInt(((Object[]) inputEnumerator.current())[0]) != 1 && ((Object[]) inputEnumerator.current())[1] != null && org.apache.calcite.runtime.SqlFunctions.toInt(((Object[]) inputEnumerator.current())[1]) != 1) {
          

          It is a bit strange that ((Object[]) inputEnumerator.current())[0] is not factored out to a local variable.

          Same for testWhereNullable:

          /*  16 */                 if ((Object) ((org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current()).commission != null && ((org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current()).commission.intValue() > 800) {

          JdbcTest.testWhereOrAndNullable is even more surprising: it creates write-only locals:

          /* 238 */             public boolean moveNext() {
          /* 239 */               while (inputEnumerator.moveNext()) {
          /* 240 */                 final Object[] current1;
          /* 241 */                 final Object[] current2;
          /* 242 */                 final Object[] current3;
          /* 243 */                 final Object[] current4;
          /* 244 */
              if (((Object[]) inputEnumerator.current())[1] != null
                  && ((Object[]) inputEnumerator.current())[0] != null && (
                  org.apache.calcite.runtime.SqlFunctions
                      .eq((current1 = (Object[]) inputEnumerator.current())[0] == null ? (String) null
                          : (current1 = (Object[]) inputEnumerator.current())[0].toString(), "a")
                      && org.apache.calcite.runtime.SqlFunctions
                      .eq((current2 = (Object[]) inputEnumerator.current())[1] == null ? (String) null
                          : (current2 = (Object[]) inputEnumerator.current())[1].toString(), "b") ||
                      org.apache.calcite.runtime.SqlFunctions
                          .eq((current3 = (Object[]) inputEnumerator.current())[0] == null ? (String) null
                              : (current3 = (Object[]) inputEnumerator.current())[0].toString(), "b")
                          && org.apache.calcite.runtime.SqlFunctions
                          .eq((current4 = (Object[]) inputEnumerator.current())[1] == null ? (String) null
                              : (current4 = (Object[]) inputEnumerator.current())[1].toString(), "c"))) {
          /* 245 */                   return true;
          /* 246 */                 }
          /* 247 */               }
          /* 248 */               return false;
          /* 249 */             }
          

          org.apache.calcite.test.JdbcTest#testAggregateEmpty
          Before:

          /* 135 */               final Object[] current = (Object[]) inputEnumerator.current();
          /* 136 */               final Object inp0_ = current[0];
          /* 137 */               final int inp1_ = org.apache.calcite.runtime.SqlFunctions.toInt(current[1]);
          /* 138 */               final long inp0_0 = org.apache.calcite.runtime.SqlFunctions.toLong(current[0]);
          /* 139 */               final boolean v0 = inp0_0 == 0L;
          /* 140 */               return new Object[] {
          /* 141 */                   inp0_,
          /* 142 */                   inp0_,
          /* 143 */                   v0 ? (Integer) null : Integer.valueOf(inp1_),
          /* 144 */                   v0 ? (Integer) null : Integer.valueOf((int) ((long) inp1_ / inp0_0))};
          

          After:

          /* 136 */             public Object current() {
          /* 137 */               final Object[] current = (Object[]) inputEnumerator.current();
          /* 138 */               final Object inp0_ = current[0];
          /* 139 */               final long inp0_0 = org.apache.calcite.runtime.SqlFunctions.toLong(current[0]);
          /* 140 */               final boolean v = inp0_0 == 0L;
          /* 141 */               return new Object[] {
          /* 142 */                   inp0_,
          /* 143 */                   inp0_,
          /* 144 */                   v ? (Integer) null : (Integer) current[1],
          /* 145 */                   v || (Integer) current[1] == null ? (Integer) null : Integer.valueOf((int) ((long) (v ? (Integer) null : (Integer) current[1]).intValue() / inp0_0))};
          /* 146 */             }

          testReuseExpressionWhenNullChecking5
          Before:

          /*  22 */             public Object current() {
          /*  23 */               final org.apache.calcite.test.JdbcTest.Employee current = (org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current();
          /*  24 */               final String inp2_ = current.name;
          /*  25 */               final int inp1_ = current.deptno;
          /*  26 */               return inp2_ == null || $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ || current.empid <= inp1_ && inp1_ * 8 <= 8 ? (String) null : org.apache.calcite.runtime.SqlFunctions.substring(org.apache.calcite.runtime.SqlFunctions.trim(true, true, " ", org.apache.calcite.runtime.SqlFunctions.substring(inp2_, inp1_ * 0 + 1)), $L4J$C$5_2);
          /*  27 */             }
          /*  28 */ 
          /*  29 */             static final boolean $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = org.apache.calcite.runtime.SqlFunctions.eq("sa", "sa");
          /*  30 */             static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = !$L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_;
          /*  31 */             static final int $L4J$C$5_2 = 5 - 2;

          After:
          Notes:
          1) Integer.valueOf(1)==null check is produced. "before" variation seemed to optimize that.
          2) : (Integer) null).intValue() - 2) at the end looks dangerous. It might produce NPE.

          /*  22 */             public Object current() {
          /*  23 */               final org.apache.calcite.test.JdbcTest.Employee current = (org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current();
          /*  24 */               return (Object) current.name == null ? (String) null : $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueabc5caed ? (String) null : (current.empid > current.deptno ? $L4J$C$Integer_valueOf_5_ : current.deptno * 8 > 8 ? $L4J$C$Integer_valueOf_5_ : (Integer) null) == null ? (String) null : org.apache.calcite.runtime.SqlFunctions.substring(org.apache.calcite.runtime.SqlFunctions.trim(true, true, " ", org.apache.calcite.runtime.SqlFunctions.substring(current.name, current.deptno * 0 + $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueO102161ff.intValue())), (current.empid > current.deptno ? $L4J$C$Integer_valueOf_5_ : current.deptno * 8 > 8 ? $L4J$C$Integer_valueOf_5_ : (Integer) null).intValue() - 2);
          /*  25 */             }
          /*  26 */ 
          /*  27 */             static final boolean $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = org.apache.calcite.runtime.SqlFunctions.eq("sa", "sa");
          /*  28 */             static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = !$L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_;
          /*  29 */             static final Integer $L4J$C$Integer_valueOf_1_ = Integer.valueOf(1);
          /*  30 */             static final boolean $L4J$C$Integer_valueOf_1_null = $L4J$C$Integer_valueOf_1_ == null;
          /*  31 */             static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueabc5caed = $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ || $L4J$C$Integer_valueOf_1_null;
          /*  32 */             static final Integer $L4J$C$Integer_valueOf_5_ = Integer.valueOf(5);
          /*  33 */             static final Integer $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueO102161ff = $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ ? $L4J$C$Integer_valueOf_1_ : (Integer) null;
          /*  34 */           };
          /*  35 */       }
          select  case when 1>2 then 10 else case when 3>4 then 20 end end-2
            from "hr"."emps"
          

          Before:

          /*  22 */             public Object current() {
          /*  23 */               final boolean v1 = $L4J$C$1_2;
          /*  24 */               return !v1 && $L4J$C$3_4 ? (Integer) null : Integer.valueOf((v1 ? 10 : 20) - 2);
          /*  25 */             }
          /*  26 */ 
          /*  27 */             static final boolean $L4J$C$1_2 = 1 > 2;
          /*  28 */             static final boolean $L4J$C$3_4 = 3 <= 4;

          After:
          Notes: strange reboxing (v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null).intValue()

          /*  22 */             public Object current() {
          /*  23 */               final boolean v = $L4J$C$1_2;
          /*  24 */               return (v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null) == null ? (Integer) null : Integer.valueOf((v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null).intValue() - 2);
          /*  25 */             }
          /*  26 */ 
          /*  27 */             static final boolean $L4J$C$1_2 = 1 > 2;
          /*  28 */             static final Integer $L4J$C$Integer_valueOf_10_ = Integer.valueOf(10);
          /*  29 */             static final boolean $L4J$C$3_4 = 3 > 4;
          /*  30 */             static final Integer $L4J$C$Integer_valueOf_20_ = Integer.valueOf(20);
          /*  31 */             static final Integer $L4J$C$3_4_Integer_valueOf_20_Integer_null = $L4J$C$3_4 ? $L4J$C$Integer_valueOf_20_ : (Integer) null;
          

          Test (testTrivialSort with slight modification of a query):

              final String sql = "select a.\"value\"||b.\"value\"\n"
                  + "  from \"bools\" a\n"
                  + "     , \"bools\" b\n"
                  + " offset 0";
              CalciteAssert.that()
                  .withSchema("s",
                      new ReflectiveSchema(
                          new ReflectiveSchemaTest.CatchallSchema()))

          Before:

          /* 101 */             public Object current() {
          /* 102 */               final Object[] current = (Object[]) inputEnumerator.current();
          /* 103 */               final String inp0_ = current[0] == null ? (String) null : current[0].toString();
          /* 104 */               final String inp1_ = current[1] == null ? (String) null : current[1].toString();
          /* 105 */               return inp0_ == null || inp1_ == null ? (String) null : org.apache.calcite.runtime.SqlFunctions.concat(inp0_, inp1_);
          /* 106 */             }
          

          After:
          Note: {{current[0] == null }} is redundant in concat argument

          /* 101 */             public Object current() {
          /* 102 */               final Object[] current = (Object[]) inputEnumerator.current();
          /* 103 */               return current[0] == null ? $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ : current[1] == null ? $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ : org.apache.calcite.runtime.SqlFunctions.concat(current[0] == null ? (String) null : current[0].toString(), current[1] == null ? (String) null : current[1].toString());
          /* 104 */             }
          /* 105 */ 
          /* 106 */             static final String $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ = org.apache.calcite.runtime.SqlFunctions.truncate((String) null, 1);
          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - JdbcTest.testBooleansInWhere: /* 252 */ if ((( Object []) inputEnumerator.current())[0] != null && org.apache.calcite.runtime.SqlFunctions.toInt((( Object []) inputEnumerator.current())[0]) != 1 && (( Object []) inputEnumerator.current())[1] != null && org.apache.calcite.runtime.SqlFunctions.toInt((( Object []) inputEnumerator.current())[1]) != 1) { It is a bit strange that ((Object[]) inputEnumerator.current())[0] is not factored out to a local variable. Same for testWhereNullable: /* 16 */ if (( Object ) ((org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current()).commission != null && ((org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current()).commission.intValue() > 800) { JdbcTest.testWhereOrAndNullable is even more surprising: it creates write-only locals: /* 238 */ public boolean moveNext() { /* 239 */ while (inputEnumerator.moveNext()) { /* 240 */ final Object [] current1; /* 241 */ final Object [] current2; /* 242 */ final Object [] current3; /* 243 */ final Object [] current4; /* 244 */ if ((( Object []) inputEnumerator.current())[1] != null && (( Object []) inputEnumerator.current())[0] != null && ( org.apache.calcite.runtime.SqlFunctions .eq((current1 = ( Object []) inputEnumerator.current())[0] == null ? ( String ) null : (current1 = ( Object []) inputEnumerator.current())[0].toString(), "a" ) && org.apache.calcite.runtime.SqlFunctions .eq((current2 = ( Object []) inputEnumerator.current())[1] == null ? ( String ) null : (current2 = ( Object []) inputEnumerator.current())[1].toString(), "b" ) || org.apache.calcite.runtime.SqlFunctions .eq((current3 = ( Object []) inputEnumerator.current())[0] == null ? ( String ) null : (current3 = ( Object []) inputEnumerator.current())[0].toString(), "b" ) && org.apache.calcite.runtime.SqlFunctions .eq((current4 = ( Object []) inputEnumerator.current())[1] == null ? ( String ) null : (current4 = ( Object []) inputEnumerator.current())[1].toString(), "c" ))) { /* 245 */ return true ; /* 246 */ } /* 247 */ } /* 248 */ return false ; /* 249 */ } org.apache.calcite.test.JdbcTest#testAggregateEmpty Before: /* 135 */ final Object [] current = ( Object []) inputEnumerator.current(); /* 136 */ final Object inp0_ = current[0]; /* 137 */ final int inp1_ = org.apache.calcite.runtime.SqlFunctions.toInt(current[1]); /* 138 */ final long inp0_0 = org.apache.calcite.runtime.SqlFunctions.toLong(current[0]); /* 139 */ final boolean v0 = inp0_0 == 0L; /* 140 */ return new Object [] { /* 141 */ inp0_, /* 142 */ inp0_, /* 143 */ v0 ? ( Integer ) null : Integer .valueOf(inp1_), /* 144 */ v0 ? ( Integer ) null : Integer .valueOf(( int ) (( long ) inp1_ / inp0_0))}; After: /* 136 */ public Object current() { /* 137 */ final Object [] current = ( Object []) inputEnumerator.current(); /* 138 */ final Object inp0_ = current[0]; /* 139 */ final long inp0_0 = org.apache.calcite.runtime.SqlFunctions.toLong(current[0]); /* 140 */ final boolean v = inp0_0 == 0L; /* 141 */ return new Object [] { /* 142 */ inp0_, /* 143 */ inp0_, /* 144 */ v ? ( Integer ) null : ( Integer ) current[1], /* 145 */ v || ( Integer ) current[1] == null ? ( Integer ) null : Integer .valueOf(( int ) (( long ) (v ? ( Integer ) null : ( Integer ) current[1]).intValue() / inp0_0))}; /* 146 */ } testReuseExpressionWhenNullChecking5 Before: /* 22 */ public Object current() { /* 23 */ final org.apache.calcite.test.JdbcTest.Employee current = (org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current(); /* 24 */ final String inp2_ = current.name; /* 25 */ final int inp1_ = current.deptno; /* 26 */ return inp2_ == null || $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ || current.empid <= inp1_ && inp1_ * 8 <= 8 ? ( String ) null : org.apache.calcite.runtime.SqlFunctions.substring(org.apache.calcite.runtime.SqlFunctions.trim( true , true , " " , org.apache.calcite.runtime.SqlFunctions.substring(inp2_, inp1_ * 0 + 1)), $L4J$C$5_2); /* 27 */ } /* 28 */ /* 29 */ static final boolean $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = org.apache.calcite.runtime.SqlFunctions.eq( "sa" , "sa" ); /* 30 */ static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = !$L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_; /* 31 */ static final int $L4J$C$5_2 = 5 - 2; After: Notes: 1) Integer.valueOf(1)==null check is produced. "before" variation seemed to optimize that. 2) : (Integer) null).intValue() - 2) at the end looks dangerous. It might produce NPE. /* 22 */ public Object current() { /* 23 */ final org.apache.calcite.test.JdbcTest.Employee current = (org.apache.calcite.test.JdbcTest.Employee) inputEnumerator.current(); /* 24 */ return ( Object ) current.name == null ? ( String ) null : $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueabc5caed ? ( String ) null : (current.empid > current.deptno ? $L4J$C$Integer_valueOf_5_ : current.deptno * 8 > 8 ? $L4J$C$Integer_valueOf_5_ : ( Integer ) null ) == null ? ( String ) null : org.apache.calcite.runtime.SqlFunctions.substring(org.apache.calcite.runtime.SqlFunctions.trim( true , true , " " , org.apache.calcite.runtime.SqlFunctions.substring(current.name, current.deptno * 0 + $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueO102161ff.intValue())), (current.empid > current.deptno ? $L4J$C$Integer_valueOf_5_ : current.deptno * 8 > 8 ? $L4J$C$Integer_valueOf_5_ : ( Integer ) null ).intValue() - 2); /* 25 */ } /* 26 */ /* 27 */ static final boolean $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = org.apache.calcite.runtime.SqlFunctions.eq( "sa" , "sa" ); /* 28 */ static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ = !$L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_; /* 29 */ static final Integer $L4J$C$Integer_valueOf_1_ = Integer .valueOf(1); /* 30 */ static final boolean $L4J$C$Integer_valueOf_1_null = $L4J$C$Integer_valueOf_1_ == null ; /* 31 */ static final boolean $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueabc5caed = $L4J$C$_org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ || $L4J$C$Integer_valueOf_1_null; /* 32 */ static final Integer $L4J$C$Integer_valueOf_5_ = Integer .valueOf(5); /* 33 */ static final Integer $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_Integer_valueO102161ff = $L4J$C$org_apache_calcite_runtime_SqlFunctions_eq_sa_sa_ ? $L4J$C$Integer_valueOf_1_ : ( Integer ) null ; /* 34 */ }; /* 35 */ } select case when 1>2 then 10 else case when 3>4 then 20 end end-2 from "hr" . "emps" Before: /* 22 */ public Object current() { /* 23 */ final boolean v1 = $L4J$C$1_2; /* 24 */ return !v1 && $L4J$C$3_4 ? ( Integer ) null : Integer .valueOf((v1 ? 10 : 20) - 2); /* 25 */ } /* 26 */ /* 27 */ static final boolean $L4J$C$1_2 = 1 > 2; /* 28 */ static final boolean $L4J$C$3_4 = 3 <= 4; After: Notes: strange reboxing (v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null).intValue() /* 22 */ public Object current() { /* 23 */ final boolean v = $L4J$C$1_2; /* 24 */ return (v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null) == null ? ( Integer ) null : Integer .valueOf((v ? $L4J$C$Integer_valueOf_10_ : $L4J$C$3_4_Integer_valueOf_20_Integer_null).intValue() - 2); /* 25 */ } /* 26 */ /* 27 */ static final boolean $L4J$C$1_2 = 1 > 2; /* 28 */ static final Integer $L4J$C$Integer_valueOf_10_ = Integer .valueOf(10); /* 29 */ static final boolean $L4J$C$3_4 = 3 > 4; /* 30 */ static final Integer $L4J$C$Integer_valueOf_20_ = Integer .valueOf(20); /* 31 */ static final Integer $L4J$C$3_4_Integer_valueOf_20_Integer_null = $L4J$C$3_4 ? $L4J$C$Integer_valueOf_20_ : ( Integer ) null ; Test (testTrivialSort with slight modification of a query): final String sql = "select a.\" value\ "||b.\" value\ "\n" + " from \" bools\ " a\n" + " , \" bools\ " b\n" + " offset 0" ; CalciteAssert.that() .withSchema( "s" , new ReflectiveSchema( new ReflectiveSchemaTest.CatchallSchema())) Before: /* 101 */ public Object current() { /* 102 */ final Object [] current = ( Object []) inputEnumerator.current(); /* 103 */ final String inp0_ = current[0] == null ? ( String ) null : current[0].toString(); /* 104 */ final String inp1_ = current[1] == null ? ( String ) null : current[1].toString(); /* 105 */ return inp0_ == null || inp1_ == null ? ( String ) null : org.apache.calcite.runtime.SqlFunctions.concat(inp0_, inp1_); /* 106 */ } After: Note: {{current [0] == null }} is redundant in concat argument /* 101 */ public Object current() { /* 102 */ final Object [] current = ( Object []) inputEnumerator.current(); /* 103 */ return current[0] == null ? $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ : current[1] == null ? $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ : org.apache.calcite.runtime.SqlFunctions.concat(current[0] == null ? ( String ) null : current[0].toString(), current[1] == null ? ( String ) null : current[1].toString()); /* 104 */ } /* 105 */ /* 106 */ static final String $L4J$C$org_apache_calcite_runtime_SqlFunctions_truncate_String_null_1_ = org.apache.calcite.runtime.SqlFunctions.truncate(( String ) null , 1);
          Hide
          julianhyde Julian Hyde added a comment -

          There is a proposed fix in https://github.com/julianhyde/calcite/commits/1066-oracle. (Yes, that branch also contains a fix for CALCITE-1066.) Please review.

          Show
          julianhyde Julian Hyde added a comment - There is a proposed fix in https://github.com/julianhyde/calcite/commits/1066-oracle . (Yes, that branch also contains a fix for CALCITE-1066 .) Please review.
          Hide
          julianhyde Julian Hyde added a comment - - edited

          Frankie Bollaert, Please state explicitly which versions you are (or are not) seeing these problems against. It sounds as if you are a couple of versions behind. Some problems have been fixed.

          And since, as you correctly surmise, we are not interested in producing maintenance releases for previous versions, I am much less interested in problems that have been fixed in the master branch.

          Show
          julianhyde Julian Hyde added a comment - - edited Frankie Bollaert , Please state explicitly which versions you are (or are not) seeing these problems against. It sounds as if you are a couple of versions behind. Some problems have been fixed. And since, as you correctly surmise, we are not interested in producing maintenance releases for previous versions, I am much less interested in problems that have been fixed in the master branch.
          Hide
          frankieb Frankie Bollaert added a comment - - edited

          Issue occurs on master branch and not on branch-1.5
          Another date problem that occurs with the CSV example:

          select * from sales.emps e WHERE "JOINEDAT" > date '2010-01-01';
          
          /* 17 */  public boolean moveNext() {
          /* 18 */    while (inputEnumerator.moveNext()) {
          /* 19 */      final Integer inp9_ = (Integer) ((Object[]) inputEnumerator.current())[9];
          /* 20 */      if (inp9_ != null && inp9_.intValue() > 14610) {
          /* 21 */        return true;
          /* 22 */      }
          /* 23 */    }
          /* 24 */    return false;
          /* 25 */  }
          
          Error: Error while executing SQL "select * from sales.emps e WHERE "JOINEDAT" > date '2010-01-01'": java.sql.Date cannot be cast to java.lang.Integer (state=,code=0)
          

          This issue was introduced by
          commit d4bbf58d6e5f4f67ebae412b5a1aae23769eccb7
          CALCITE-1041 User-defined function that returns DATE or TIMESTAMP value
          Reverting this commit makes the problem disappear.

          Should I make a separate JIRA ticket for this?

          Show
          frankieb Frankie Bollaert added a comment - - edited Issue occurs on master branch and not on branch-1.5 Another date problem that occurs with the CSV example: select * from sales.emps e WHERE "JOINEDAT" > date '2010-01-01'; /* 17 */ public boolean moveNext() { /* 18 */ while (inputEnumerator.moveNext()) { /* 19 */ final Integer inp9_ = ( Integer ) (( Object []) inputEnumerator.current())[9]; /* 20 */ if (inp9_ != null && inp9_.intValue() > 14610) { /* 21 */ return true ; /* 22 */ } /* 23 */ } /* 24 */ return false ; /* 25 */ } Error: Error while executing SQL "select * from sales.emps e WHERE " JOINEDAT " > date '2010-01-01'" : java.sql.Date cannot be cast to java.lang. Integer (state=,code=0) This issue was introduced by commit d4bbf58d6e5f4f67ebae412b5a1aae23769eccb7 CALCITE-1041 User-defined function that returns DATE or TIMESTAMP value Reverting this commit makes the problem disappear. Should I make a separate JIRA ticket for this?
          Hide
          frankieb Frankie Bollaert added a comment - - edited

          Issue occurs on master and https://github.com/julianhyde/calcite/tree/1054-timestamp-npe. This is merely more detailed logging of the test cases you provided.

          toIntOptional() is called and next intValue(). So intValue() is called on null when a given field in a record is null.

          /* 51 */  public boolean moveNext() {
          /* 52 */    while (inputEnumerator.moveNext()) {
          /* 53 */      final java.sql.Date inp16_ = ((org.apache.calcite.test.ReflectiveSchemaTest.EveryType) inputEnumerator.current()).sqlDate;
          /* 54 */      final int v0 = org.apache.calcite.runtime.SqlFunctions.toIntOptional(inp16_).intValue();
          /* 55 */      if (inp16_ != null && v0 > 0 && (inp16_ != null && v0 < 3652)) { 
          /* 56 */        return true;
          /* 57 */      }
          /* 58 */    }
          /* 59 */    return false;
          /* 60 */  }
          

          Stack trace snippet:

          > Caused by: java.lang.NullPointerException
          > 	at Baz$1$1.moveNext(ANONYMOUS.java:54)
          > 	at org.apache.calcite.linq4j.EnumerableDefaults.aggregate(EnumerableDefaults.java:112)
          > 	at org.apache.calcite.linq4j.DefaultEnumerable.aggregate(DefaultEnumerable.java:107)
          > 	at Baz.bind(Baz.java:96)
          

          Earlier you made the comment:

          with the map set so that isKnownNullable returns FALSE


          I think this refers to the root cause.

          Show
          frankieb Frankie Bollaert added a comment - - edited Issue occurs on master and https://github.com/julianhyde/calcite/tree/1054-timestamp-npe . This is merely more detailed logging of the test cases you provided. toIntOptional() is called and next intValue(). So intValue() is called on null when a given field in a record is null. /* 51 */ public boolean moveNext() { /* 52 */ while (inputEnumerator.moveNext()) { /* 53 */ final java.sql.Date inp16_ = ((org.apache.calcite.test.ReflectiveSchemaTest.EveryType) inputEnumerator.current()).sqlDate; /* 54 */ final int v0 = org.apache.calcite.runtime.SqlFunctions.toIntOptional(inp16_).intValue(); /* 55 */ if (inp16_ != null && v0 > 0 && (inp16_ != null && v0 < 3652)) { /* 56 */ return true ; /* 57 */ } /* 58 */ } /* 59 */ return false ; /* 60 */ } Stack trace snippet: > Caused by: java.lang.NullPointerException > at Baz$1$1.moveNext(ANONYMOUS.java:54) > at org.apache.calcite.linq4j.EnumerableDefaults.aggregate(EnumerableDefaults.java:112) > at org.apache.calcite.linq4j.DefaultEnumerable.aggregate(DefaultEnumerable.java:107) > at Baz.bind(Baz.java:96) Earlier you made the comment: with the map set so that isKnownNullable returns FALSE I think this refers to the root cause.
          Hide
          julianhyde Julian Hyde added a comment -

          See e.g. JdbcTest.testRunMisc.

          Show
          julianhyde Julian Hyde added a comment - See e.g. JdbcTest.testRunMisc.
          Hide
          frankieb Frankie Bollaert added a comment -

          How do you execute these .iq files?
          Is it by copy/pasting the statements in a terminal?

          Show
          frankieb Frankie Bollaert added a comment - How do you execute these .iq files? Is it by copy/pasting the statements in a terminal?
          Hide
          julianhyde Julian Hyde added a comment -

          Unfortunately it still seems to be an issue on master. See https://github.com/julianhyde/calcite/tree/1054-timestamp-npe, a branch containing a test case.

          Show
          julianhyde Julian Hyde added a comment - Unfortunately it still seems to be an issue on master. See https://github.com/julianhyde/calcite/tree/1054-timestamp-npe , a branch containing a test case.
          Hide
          frankieb Frankie Bollaert added a comment - - edited

          Thank you Julian, your pointers are very helpful.

          The snippets of generated code in this issue where created using Calcite 1.4. Using the code of the git master branch produces a different result. If you are not planning any 1.4 or 1.5 (which generates similar code) branch releases, then this issue is not relevant, and may best be closed.

          Show
          frankieb Frankie Bollaert added a comment - - edited Thank you Julian, your pointers are very helpful. The snippets of generated code in this issue where created using Calcite 1.4. Using the code of the git master branch produces a different result. If you are not planning any 1.4 or 1.5 (which generates similar code) branch releases, then this issue is not relevant, and may best be closed.
          Hide
          julianhyde Julian Hyde added a comment -

          The code generation occurs within RexToLixTranslator. For types that have primitive representation (such as integer) we insert an 'is not null' check and then we re-invoke the code generation with the map set so that isKnownNullable returns FALSE, in which case we can use the primitive representation.

          Date, Time, Timestamp are represented as primitive types (int, int, long) but only after they have been unwrapped. Maybe this unwrapping is taking place at the wrong time.

          Show
          julianhyde Julian Hyde added a comment - The code generation occurs within RexToLixTranslator. For types that have primitive representation (such as integer) we insert an 'is not null' check and then we re-invoke the code generation with the map set so that isKnownNullable returns FALSE, in which case we can use the primitive representation. Date, Time, Timestamp are represented as primitive types (int, int, long) but only after they have been unwrapped. Maybe this unwrapping is taking place at the wrong time.

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              frankieb Frankie Bollaert
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development