Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-3498

Unnest operation's ordinality should be deterministic

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.21.0
    • Fix Version/s: 1.22.0
    • Component/s: core
    • Environment:

      **

      Description

      Current now, the compution of unnest operation's ordinality is conducted in current() method ( https://github.com/apache/calcite/blob/3853118b4d1d48f7d3970b5488baf7ca78d5028e/core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java#L2764).

      Consequently, when the method is called, the ordinality will be incremented, leading to incorrect and un-deterministic result.

      We can use a simple query for illustration.

      Query:
      ==============================================================
      select v, o
      from unnest(array[100,200]) with ordinality as t1(v, o)
      where v > 1
      
      Expected Result
      ==============================================================
      V=100; O=1
      V=200; O=2
      

      However, we get the follow incorrect result.

      V=100; O=2
      V=200; O=4
      

      We can infer to the code generated. It can be seen that inputEnumerator.current() has been called two times, one for filter (v>1) and one for select (v, o).

      public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
        final org.apache.calcite.linq4j.Enumerable _inputEnumerable = ..............
        final org.apache.calcite.linq4j.Enumerable _inputEnumerable0 = child.selectMany(org.apache.calcite.runtime.SqlFunctions.flatProduct(new int[] {
          -1}, true, new org.apache.calcite.runtime.SqlFunctions.FlatProductInputType[] {
          org.apache.calcite.runtime.SqlFunctions.FlatProductInputType.SCALAR}));
        return new org.apache.calcite.linq4j.AbstractEnumerable(){
            public org.apache.calcite.linq4j.Enumerator enumerator() {
              return new org.apache.calcite.linq4j.Enumerator(){
                  public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable0.enumerator();
                  public void reset() {
                    inputEnumerator.reset();
                  }            public boolean moveNext() {
                    while (inputEnumerator.moveNext()) {
                      if (org.apache.calcite.runtime.SqlFunctions.toInt(((org.apache.calcite.runtime.FlatLists.ComparableList) inputEnumerator.current()).get(0)) > 1) {
                        return true;
                      }
                    }
                    return false;
                  }            public void close() {
                    inputEnumerator.close();
                  }            public Object current() {
                    final org.apache.calcite.runtime.FlatLists.ComparableList current = (org.apache.calcite.runtime.FlatLists.ComparableList) inputEnumerator.current();
                    return new Object[] {
                        current.get(0),
                        current.get(1)};
                  }          };
            }    };
      }
      public Class getElementType() {
        return java.lang.Object[].class;
      }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                donnyzone Feng Zhu
                Reporter:
                donnyzone Feng Zhu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h