Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-542

Support for Aggregate with grouping sets in RelMdColumnOrigins

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0-incubating
    • Component/s: None
    • Labels:
      None

      Description

      When we call getColumnOrigins on an Aggregate operator with grouping sets, an error arrises if we are trying to derive the origin of an indicator column associated to a grouping column.

        Activity

        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment -

        I submitted a pull request to fix this error, as it is blocking HIVE-8988. Julian Hyde, what do you think?

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - I submitted a pull request to fix this error, as it is blocking HIVE-8988 . Julian Hyde , what do you think?
        Hide
        julianhyde Julian Hyde added a comment -

        I added a review comment. Can you also add a test that ResultSetMetaData.getColumnName(int) works for columns in a GROUPING SETS query. I believe that info in the JDBC driver comes from RelMdColumnOrigins. TableInRootSchemaTest.testAddingTableInRootSchema does something similar to this.

        Show
        julianhyde Julian Hyde added a comment - I added a review comment. Can you also add a test that ResultSetMetaData.getColumnName(int) works for columns in a GROUPING SETS query. I believe that info in the JDBC driver comes from RelMdColumnOrigins. TableInRootSchemaTest.testAddingTableInRootSchema does something similar to this.
        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment -

        Thanks! I just changed the line according to your review. I also added a test (RelMdColumnOriginsTest) based on the query that was producing the problem in Hive: the test does not pass, but I think the problem is not related to RelMdColumnOrigins (is it a problem related to the data type of the grouping(col) column?).

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - Thanks! I just changed the line according to your review. I also added a test (RelMdColumnOriginsTest) based on the query that was producing the problem in Hive: the test does not pass, but I think the problem is not related to RelMdColumnOrigins (is it a problem related to the data type of the grouping(col) column?).
        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - - edited

        Just to clarify, the problem in the test query:

        SELECT TABLE1.ID, TABLE2.ID FROM
         (SELECT GROUPING(A) AS ID FROM T1 
          GROUP BY ROLLUP(A,B)) TABLE1 
        JOIN 
         (SELECT GROUPING(A) AS ID FROM T1 
          GROUP BY ROLLUP(A,B)) TABLE2 
        ON TABLE1.ID = TABLE2.ID
        

        is in the results, not in the metadata. In particular:

        Expected: <[ID=1; ID=1, ID=0; ID=0]>
             but: was <[ID=1; ID=1, ID=0; ID=0 x 25]>
        

        If you agree, I can skip the results check in this test, and open a new JIRA for this.

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - - edited Just to clarify, the problem in the test query: SELECT TABLE1.ID, TABLE2.ID FROM (SELECT GROUPING(A) AS ID FROM T1 GROUP BY ROLLUP(A,B)) TABLE1 JOIN (SELECT GROUPING(A) AS ID FROM T1 GROUP BY ROLLUP(A,B)) TABLE2 ON TABLE1.ID = TABLE2.ID is in the results, not in the metadata. In particular: Expected: <[ID=1; ID=1, ID=0; ID=0]> but: was <[ID=1; ID=1, ID=0; ID=0 x 25]> If you agree, I can skip the results check in this test, and open a new JIRA for this.
        Hide
        julianhyde Julian Hyde added a comment -

        The actual results look OK. Isn't it right that you get (0, 0) repeating 25 times and (1, 1) once. Your expected result has (0, 0) only once, and that looks wrong.

        Note that the 'x 25' comes from how ImmutableMultiSet.toString works when there are repeats in the set. It's not data!

        Show
        julianhyde Julian Hyde added a comment - The actual results look OK. Isn't it right that you get (0, 0) repeating 25 times and (1, 1) once. Your expected result has (0, 0) only once, and that looks wrong. Note that the 'x 25' comes from how ImmutableMultiSet.toString works when there are repeats in the set. It's not data!
        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment -

        OK, now I get it, thanks! I'm uploading a new patch.

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - OK, now I get it, thanks! I'm uploading a new patch.
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/4e95a8ca .
        Hide
        julianhyde Julian Hyde added a comment -

        Closing now that 1.0.0-incubating has been released.

        Show
        julianhyde Julian Hyde added a comment - Closing now that 1.0.0-incubating has been released.

          People

          • Assignee:
            jcamachorodriguez Jesus Camacho Rodriguez
            Reporter:
            jcamachorodriguez Jesus Camacho Rodriguez
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development