Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1212

Cassandra Projections throw an NPE on a join with another non-cassandra table

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.8.0
    • Component/s: cassandra
    • Labels:
      None

      Description

      Okay, I am really note entirely sure where to begin describing this, since it involves a lot of what I don't fully understand but Ill list out some of the observations and maybe that might help.

      I have a Jdbc table named assets, it has a field named name that is VARCHAR. If I attempt to do a join like so

      select * 
      from core.assets
      join twissandra.tweets
      on core.assets.name = twissandra.tweets.username
      

      I get an exception like this

      java.lang.NullPointerException: null
      	at org.apache.calcite.adapter.cassandra.CassandraTable.query(CassandraTable.java:121)
      	at org.apache.calcite.adapter.cassandra.CassandraTable$CassandraQueryable.query(CassandraTable.java:206)
      	at Baz.bind(Baz.java:6)
      	at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:327)
      
      

      It looks like it's related to the username field potentially not having the same type CHAR vs VARCHAR so it gets flagged as a new field? This code seems to be adding four fields instead of five the additional username4 which causes the NPE.

      I have the output of the code generation here if that helps

      /*   1 */ org.apache.calcite.DataContext root;
      /*   2 */ 
      /*   3 */ public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root0) {
      /*   4 */   root = root0;
      /*   5 */   final java.util.List predicates = java.util.Arrays.asList(new String[] {});
      /*   6 */   final org.apache.calcite.linq4j.Enumerable _inputEnumerable = ((org.apache.calcite.adapter.cassandra.CassandraTable.CassandraQueryable) org.apache.calcite.schema.Schemas.queryable(root, root.getRootSchema().getSubSchema("twissandra"), java.lang.Object[].class, "tweets")).query(java.util.Arrays.asList(new org.apache.calcite.util.Pair[] {
      /*   7 */     new org.apache.calcite.util.Pair(
      /*   8 */       "tweet_id",
      /*   9 */       java.lang.String.class),
      /*  10 */     new org.apache.calcite.util.Pair(
      /*  11 */       "body",
      /*  12 */       java.lang.String.class),
      /*  13 */     new org.apache.calcite.util.Pair(
      /*  14 */       "tenant_id",
      /*  15 */       java.lang.String.class),
      /*  16 */     new org.apache.calcite.util.Pair(
      /*  17 */       "username",
      /*  18 */       java.lang.String.class),
      /*  19 */     new org.apache.calcite.util.Pair(
      /*  20 */       "username4",
      /*  21 */       java.lang.String.class)}), predicates, predicates, null).join(org.apache.calcite.runtime.ResultSetEnumerable.of(((org.apache.calcite.adapter.jdbc.JdbcSchema) root.getRootSchema().getSubSchema("core").unwrap(org.apache.calcite.adapter.jdbc.JdbcSchema.class)).getDataSource(), "SELECT *\nFROM \"assets\"", new org.apache.calcite.linq4j.function.Function1() {
      /*  22 */     public org.apache.calcite.linq4j.function.Function0 apply(final java.sql.ResultSet resultSet) {
      /*  23 */       return new org.apache.calcite.linq4j.function.Function0() {
      /*  24 */           public Object apply() {
      /*  25 */             try {
      /*  26 */               final Object[] values = new Object[4];
      /*  27 */               values[0] = resultSet.getObject(1);
      /*  28 */               values[1] = resultSet.getObject(2);
      /*  29 */               values[2] = resultSet.getObject(3);
      /*  30 */               values[3] = resultSet.getObject(4);
      /*  31 */               return values;
      /*  32 */             } catch (java.sql.SQLException e) {
      /*  33 */               throw new RuntimeException(
      /*  34 */                 e);
      /*  35 */             }
      /*  36 */           }
      /*  37 */         }
      /*  38 */       ;
      /*  39 */     }
      /*  40 */     public Object apply(final Object resultSet) {
      /*  41 */       return apply(
      /*  42 */         (java.sql.ResultSet) resultSet);
      /*  43 */     }
      /*  44 */   }
      /*  45 */   ), new org.apache.calcite.linq4j.function.Function1() {
      /*  46 */     public String apply(Object[] v1) {
      /*  47 */       return v1[4] == null ? (String) null : v1[4].toString();
      /*  48 */     }
      /*  49 */     public Object apply(Object v1) {
      /*  50 */       return apply(
      /*  51 */         (Object[]) v1);
      /*  52 */     }
      /*  53 */   }
      /*  54 */   , new org.apache.calcite.linq4j.function.Function1() {
      /*  55 */     public String apply(Object[] v1) {
      /*  56 */       return v1[2] == null ? (String) null : v1[2].toString();
      /*  57 */     }
      /*  58 */     public Object apply(Object v1) {
      /*  59 */       return apply(
      /*  60 */         (Object[]) v1);
      /*  61 */     }
      /*  62 */   }
      /*  63 */   , new org.apache.calcite.linq4j.function.Function2() {
      /*  64 */     public Object[] apply(Object[] left, Object[] right) {
      /*  65 */       return new Object[] {
      /*  66 */           left[0],
      /*  67 */           left[1],
      /*  68 */           left[2],
      /*  69 */           left[3],
      /*  70 */           left[4],
      /*  71 */           right[0],
      /*  72 */           right[1],
      /*  73 */           right[2],
      /*  74 */           right[3]};
      /*  75 */     }
      /*  76 */     public Object[] apply(Object left, Object right) {
      /*  77 */       return apply(
      /*  78 */         (Object[]) left,
      /*  79 */         (Object[]) right);
      /*  80 */     }
      /*  81 */   }
      /*  82 */   , null, false, false);
      /*  83 */   return new org.apache.calcite.linq4j.AbstractEnumerable(){
      /*  84 */       public org.apache.calcite.linq4j.Enumerator enumerator() {
      /*  85 */         return new org.apache.calcite.linq4j.Enumerator(){
      /*  86 */             public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
      /*  87 */             public void reset() {
      /*  88 */               inputEnumerator.reset();
      /*  89 */             }
      /*  90 */ 
      /*  91 */             public boolean moveNext() {
      /*  92 */               return inputEnumerator.moveNext();
      /*  93 */             }
      /*  94 */ 
      /*  95 */             public void close() {
      /*  96 */               inputEnumerator.close();
      /*  97 */             }
      /*  98 */ 
      /*  99 */             public Object current() {
      /* 100 */               final Object[] current = (Object[]) inputEnumerator.current();
      /* 101 */               return new Object[] {
      /* 102 */                   current[5],
      /* 103 */                   current[6],
      /* 104 */                   current[7],
      /* 105 */                   current[8],
      /* 106 */                   current[0],
      /* 107 */                   current[1],
      /* 108 */                   current[2],
      /* 109 */                   current[3]};
      /* 110 */             }
      /* 111 */ 
      /* 112 */           };
      /* 113 */       }
      /* 114 */ 
      /* 115 */     };
      /* 116 */ }
      /* 117 */ 
      /* 118 */ 
      /* 119 */ public Class getElementType() {
      /* 120 */   return java.lang.Object[].class;
      /* 121 */ }
      /* 122 */ 
      /* 123 */ 
      

      Something else I noticed is that why does predicate show up twice?

      Happy to send more details, but thought I'd dump all I have.

        Issue Links

          Activity

          Hide
          michaelmior Michael Mior added a comment -

          Thanks for sharing! This looks like enough info for me to start digging into this.

          Show
          michaelmior Michael Mior added a comment - Thanks for sharing! This looks like enough info for me to start digging into this.
          Hide
          dispalt Dan Di Spaltro added a comment -

          I also noticed that you don't ever use selectFields on the implementor anywhere.

          Thanks for the help!

          Show
          dispalt Dan Di Spaltro added a comment - I also noticed that you don't ever use selectFields on the implementor anywhere. Thanks for the help!
          Hide
          dispalt Dan Di Spaltro added a comment -

          Here's an even simpler way to reproduce the issue.

          select "tweet_id", 'foo' as "bar" from "userline" where "username" = '!PUBLIC!'
          

          Do you think this adapter needs to do something similar to this?

          Show
          dispalt Dan Di Spaltro added a comment - Here's an even simpler way to reproduce the issue. select "tweet_id" , 'foo' as "bar" from "userline" where "username" = '!PUBLIC!' Do you think this adapter needs to do something similar to this ?
          Hide
          michaelmior Michael Mior added a comment -

          The main issue is that the adapter currently isn't smart enough to know that it can't push down the bar column. I'm working on a fix and I should be able to have something out this weekend.

          Show
          michaelmior Michael Mior added a comment - The main issue is that the adapter currently isn't smart enough to know that it can't push down the bar column. I'm working on a fix and I should be able to have something out this weekend.
          Hide
          michaelmior Michael Mior added a comment -

          If you get the chance, it would be great if you could try out this branch and see if it fixes your problem.

          Show
          michaelmior Michael Mior added a comment - If you get the chance, it would be great if you could try out this branch and see if it fixes your problem.
          Hide
          dispalt Dan Di Spaltro added a comment -

          It works, thank you very much!

          Show
          dispalt Dan Di Spaltro added a comment - It works, thank you very much!
          Hide
          michaelmior Michael Mior added a comment -

          No problem! Thanks for your help sorting this out

          Show
          michaelmior Michael Mior added a comment - No problem! Thanks for your help sorting this out
          Hide
          michaelmior Michael Mior added a comment -

          Fixed in 6baa9c4.

          Show
          michaelmior Michael Mior added a comment - Fixed in 6baa9c4 .
          Hide
          jni Jinfeng Ni added a comment -

          Did commit 6baa9c4bed1100012ebf4ef9547cbb077f2edaaf break the build?

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project calcite-cassandra: Compilation failure
          [ERROR] /Users/jni/work/optiq/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraTable.java:[151,41] <anonymous org.apache.calcite.adapter.cassandra.CassandraTable$2$1> is not abstract and does not override abstract method remove() in java.util.Iterator
          [ERROR] -> [Help 1]
          [ERROR]
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR]
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR]
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR] mvn <goals> -rf :calcite-cassandra
          Administrators-MacBook-Pro-53:optiq jni$ git log
          commit 6baa9c4bed1100012ebf4ef9547cbb077f2edaaf
          Author: Michael Mior <mmior@uwaterloo.ca>
          Date: Mon Apr 25 20:04:16 2016 -0400

          CALCITE-1212 Fix NPE on some Cassandra projects

          Show
          jni Jinfeng Ni added a comment - Did commit 6baa9c4bed1100012ebf4ef9547cbb077f2edaaf break the build? [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project calcite-cassandra: Compilation failure [ERROR] /Users/jni/work/optiq/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraTable.java: [151,41] <anonymous org.apache.calcite.adapter.cassandra.CassandraTable$2$1> is not abstract and does not override abstract method remove() in java.util.Iterator [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :calcite-cassandra Administrators-MacBook-Pro-53:optiq jni$ git log commit 6baa9c4bed1100012ebf4ef9547cbb077f2edaaf Author: Michael Mior <mmior@uwaterloo.ca> Date: Mon Apr 25 20:04:16 2016 -0400 CALCITE-1212 Fix NPE on some Cassandra projects
          Hide
          michaelmior Michael Mior added a comment -

          Looks like it did break the build indeed. This is odd since I didn't get this issue locally. Easy fix though. Will push shortly.

          Show
          michaelmior Michael Mior added a comment - Looks like it did break the build indeed. This is odd since I didn't get this issue locally. Easy fix though. Will push shortly.
          Hide
          michaelmior Michael Mior added a comment -

          Should be resolved by CALCITE-1215.

          Show
          michaelmior Michael Mior added a comment - Should be resolved by CALCITE-1215 .

            People

            • Assignee:
              michaelmior Michael Mior
              Reporter:
              dispalt Dan Di Spaltro
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development