Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2000

UNNEST a collection that has a field with nested data generates an Exception

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.27.0
    • None


      When unnesting a column that has another level of nesting it generates the following exception:

      Caused by: java.lang.ArrayStoreException
      	at java.lang.System.arraycopy(Native Method)
      	at org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
      	at org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
      	at org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
      	at org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
      	at Baz$4$1.moveNext(Unknown Source)
      	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<init>(Linq4j.java:680)
      	at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
      	at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
      	at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
      	at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
      	at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
      	at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
      	at org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
      	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
      	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
      	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
      	... 29 more

      The problem is that Calcite uses FlatLists internally that expect items to implement Comparable.
      However, a column with a nested collection will have an array or a List as the column value, which do not implement Comparable, so it generates the Exception.
      To test this in JdbcTest, I created a field in Employee with a collection.
      For simplicity, I created a field called dependents in Employee class:

      public static class HrSchema {
          @Override public String toString() {
            return "HrSchema";
          public final Dependent[] dependents = {
              new Dependent(10, "Michael"),
              new Dependent(10, "Jane"),
          public final Employee[] emps = {
            new Employee(100, 10, "Bill", 10000, 1000, Arrays.asList(dependents)),
            new Employee(200, 20, "Eric", 8000, 500,  Collections.<Dependent>emptyList()),
            new Employee(150, 10, "Sebastian", 7000, null,  Collections.<Dependent>emptyList()),
            new Employee(110, 10, "Theodore", 11500, 250,  Collections.<Dependent>emptyList()),
          public final Department[] depts = {
            new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
                new Location(-122, 38)),
            new Department(30, "Marketing", Collections.<Employee>emptyList(),
                new Location(0, 52)),
            new Department(40, "HR", Collections.singletonList(emps[1]), null),
          public final Dependent[] locations = {
            new Dependent(10, "San Francisco"),
            new Dependent(20, "San Diego"),
          public QueryableTable foo(int count) {
            return Smalls.generateStrings(count);
          public TranslatableTable view(String s) {
            return Smalls.view(s);
        public static class Employee {
          public final int empid;
          public final int deptno;
          public final String name;
          public final float salary;
          public final Integer commission;
          public final List<Dependent> dependents;
          public Employee(int empid, int deptno, String name, float salary,
              Integer commission) {
            this(empid, deptno, name, salary, commission, Collections.<Dependent>emptyList());
          public Employee(int empid, int deptno, String name, float salary,
              Integer commission, List<Dependent> dependents) {
            this.empid = empid;
            this.deptno = deptno;
            this.name = name;
            this.salary = salary;
            this.commission = commission;
            this.dependents = dependents;
          @Override public String toString() {
            return "Employee [empid: " + empid + ", deptno: " + deptno
                + ", name: " + name + "]";
          @Override public boolean equals(Object obj) {
            return obj == this
                || obj instanceof Employee
                && empid == ((Employee) obj).empid;

      Just running the test case JdbcTest.testUnnestArrayColumn generates the exception.


        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            rubenql Ruben Q L
            lfkauer Luis Fernando Kauer
            0 Vote for this issue
            3 Start watching this issue



              Time Tracking

                Original Estimate - Not Specified
                Not Specified
                Remaining Estimate - 0h
                Time Spent - 3h 10m
                3h 10m


                  Issue deployment