Apache Drill
  1. Apache Drill
  2. DRILL-57

Add a logical operator that returns a constant result, similar to VALUES operator in SQL

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0-milestone-1
    • Labels:
      None

      Description

      Add a logical operator that returns a constant result. This is necessary to be able to create values out of thin air, if you don't know what tables exist.

      This operator is analogous to the VALUES operator in SQL. For example,

      VALUES (1, 'a'), (2, 'b'), (3, 'c')

      returns 3 rows with 2 columns and

      VALUES (1, 'a'), (2, 'b'), (3, 'c') AS t(c1, c2)

      allows you to name those columns "c1" and "c2". VALUES is useful for evaluating expressions that don't belong in a table, for example

      VALUES 10 + 3

      (Some databases would express this "SELECT 10 + 3". Same idea, and same underlying relational operator.)

      The JSON for this new operator might look like this:

      {
      op: "constant",
      content: { [

      {c1: 1, c2: "a"}

      ,

      {c1: 2, c3: ["x", "y", "z"]}

      ] }
      }

      This operator is necessary in order to implement the SQL VALUES clause.

      1. constant.patch
        16 kB
        Jason Altekruse
      2. types_meeting_6_5_13.txt
        4 kB
        Jason Altekruse

        Activity

        Jacques Nadeau made changes -
        Target Version/s M1 [ 12324669 ]
        Fix Version/s M1 [ 12324669 ]
        Jacques Nadeau made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Jake Farrell made changes -
        Workflow jira [ 12774183 ] no-reopen-closed, patch-avail [ 12807450 ]
        Hide
        Julian Hyde added a comment -

        Yay! I'll enable the corresponding bits in sqlparser.

        Show
        Julian Hyde added a comment - Yay! I'll enable the corresponding bits in sqlparser.
        Jacques Nadeau made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Jacques Nadeau added a comment -

        Added in 97eb07ac84a106345e169e3fbf7588896d4d99c1 in mster

        Show
        Jacques Nadeau added a comment - Added in 97eb07ac84a106345e169e3fbf7588896d4d99c1 in mster
        Hide
        Jason Altekruse added a comment -

        A google doc with probably wok better for gathering opinions, here is a link:

        https://docs.google.com/document/d/1nn8sxcuBvpAHm-BoreCWELPIQ8QhhsAUSENX0s5sSys/edit

        Show
        Jason Altekruse added a comment - A google doc with probably wok better for gathering opinions, here is a link: https://docs.google.com/document/d/1nn8sxcuBvpAHm-BoreCWELPIQ8QhhsAUSENX0s5sSys/edit
        Jason Altekruse made changes -
        Attachment types_meeting_6_5_13.txt [ 12586382 ]
        Hide
        Jason Altekruse added a comment -

        Following a short meeting about type specification in JSON I have prepared a document with a description with some of our options and attached it to this issue. Please take a look at it and share your opinions of the various options or suggest some improvements.

        Show
        Jason Altekruse added a comment - Following a short meeting about type specification in JSON I have prepared a document with a description with some of our options and attached it to this issue. Please take a look at it and share your opinions of the various options or suggest some improvements.
        Hide
        Julian Hyde added a comment -

        Oops. My bad. I have corrected "returns 2 rows with 3 columns" to "returns 3 rows with 2 columns" in the description.

        Show
        Julian Hyde added a comment - Oops. My bad. I have corrected "returns 2 rows with 3 columns" to "returns 3 rows with 2 columns" in the description.
        Julian Hyde made changes -
        Description Add a logical operator that returns a constant result. This is necessary to be able to create values out of thin air, if you don't know what tables exist.

        This operator is analogous to the VALUES operator in SQL. For example,

          VALUES (1, 'a'), (2, 'b'), (3, 'c')

        returns 2 rows with 3 columns and

          VALUES (1, 'a'), (2, 'b'), (3, 'c') AS t(c1, c2)

        allows you to name those columns "c1" and "c2". VALUES is useful for evaluating expressions that don't belong in a table, for example

          VALUES 10 + 3

        (Some databases would express this "SELECT 10 + 3". Same idea, and same underlying relational operator.)

        The JSON for this new operator might look like this:

        {
        op: "constant",
                      content: { [
                        {c1: 1, c2: "a"},
                        {c1: 2, c3: ["x", "y", "z"]}
                      ] }
        }

        This operator is necessary in order to implement the SQL VALUES clause.
        Add a logical operator that returns a constant result. This is necessary to be able to create values out of thin air, if you don't know what tables exist.

        This operator is analogous to the VALUES operator in SQL. For example,

          VALUES (1, 'a'), (2, 'b'), (3, 'c')

        returns 3 rows with 2 columns and

          VALUES (1, 'a'), (2, 'b'), (3, 'c') AS t(c1, c2)

        allows you to name those columns "c1" and "c2". VALUES is useful for evaluating expressions that don't belong in a table, for example

          VALUES 10 + 3

        (Some databases would express this "SELECT 10 + 3". Same idea, and same underlying relational operator.)

        The JSON for this new operator might look like this:

        {
        op: "constant",
                      content: { [
                        {c1: 1, c2: "a"},
                        {c1: 2, c3: ["x", "y", "z"]}
                      ] }
        }

        This operator is necessary in order to implement the SQL VALUES clause.
        Jason Altekruse made changes -
        Attachment constant.patch [ 12586358 ]
        Hide
        Jason Altekruse added a comment -

        Yes that is the way that it is implemented, I noticed the typo as I was reading the description above.

        Show
        Jason Altekruse added a comment - Yes that is the way that it is implemented, I noticed the typo as I was reading the description above.
        Hide
        bfeifarek@q.com added a comment -

        Hi Jason,

        My expectation is that this returns 3 rows with 2 columns.

        Brian
        ----- Original Message -----
        From: "Jason Altekruse (JIRA)" <jira@apache.org>
        To: drill-dev@incubator.apache.org
        Sent: Wednesday, June 5, 2013 11:38:20 AM
        Subject: [jira] [Work started] (DRILL-57) Add a logical operator that returns a constant result, similar to VALUES operator in SQL

        [ https://issues.apache.org/jira/browse/DRILL-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

        Work on DRILL-57 started by Jason Altekruse.


        This message is automatically generated by JIRA.
        If you think it was sent incorrectly, please contact your JIRA administrators
        For more information on JIRA, see: http://www.atlassian.com/software/jira

        Show
        bfeifarek@q.com added a comment - Hi Jason, My expectation is that this returns 3 rows with 2 columns. Brian ----- Original Message ----- From: "Jason Altekruse (JIRA)" <jira@apache.org> To: drill-dev@incubator.apache.org Sent: Wednesday, June 5, 2013 11:38:20 AM Subject: [jira] [Work started] ( DRILL-57 ) Add a logical operator that returns a constant result, similar to VALUES operator in SQL [ https://issues.apache.org/jira/browse/DRILL-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on DRILL-57 started by Jason Altekruse. – This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
        Jason Altekruse made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Jason Altekruse made changes -
        Due Date 06/Jun/13
        Hide
        Jacques Nadeau added a comment -

        Please post what you have so far. The we can discuss type management.

        Show
        Jacques Nadeau added a comment - Please post what you have so far. The we can discuss type management.
        Jacques Nadeau made changes -
        Assignee Michael Hausenblas [ mhausenblas ] Jason Altekruse [ jaltekruse ]
        Hide
        Jason Altekruse added a comment -

        Created two new classes Constant which extends from SourceOperator and ConstantROP. Leveraged existing code in JSONRecordReader to convert JSON to UbackedRecords.

        Show
        Jason Altekruse added a comment - Created two new classes Constant which extends from SourceOperator and ConstantROP. Leveraged existing code in JSONRecordReader to convert JSON to UbackedRecords.
        Hide
        Jason Altekruse added a comment -

        I have created an implementation of this operator using the code from the JsonRecordReader to convert the records passed in the content field into unbacked records. It looks like type inferencing is working well with that, but we will need to develop a format for specifying types if we would like the option to type them explicitly.

        Show
        Jason Altekruse added a comment - I have created an implementation of this operator using the code from the JsonRecordReader to convert the records passed in the content field into unbacked records. It looks like type inferencing is working well with that, but we will need to develop a format for specifying types if we would like the option to type them explicitly.
        Hide
        Michael Hausenblas added a comment -

        I've now added the syntax definition proposal and example to the DLP http://j.mp/apache-drill-plan-syntax pls review and comment for our today's meeting, see also meeting notes at http://j.mp/apache-drill-hangouts

        Show
        Michael Hausenblas added a comment - I've now added the syntax definition proposal and example to the DLP http://j.mp/apache-drill-plan-syntax pls review and comment for our today's meeting, see also meeting notes at http://j.mp/apache-drill-hangouts
        Hide
        Jacques Nadeau added a comment -

        I'd suggest by coming up with a syntax definition for the syntax doc. Once we get agreement there, you could work on a new implementation in the org.apache.drill.common.logical.data package inside the common module. From there, build a reference implementation in the exec/ref module.

        Show
        Jacques Nadeau added a comment - I'd suggest by coming up with a syntax definition for the syntax doc. Once we get agreement there, you could work on a new implementation in the org.apache.drill.common.logical.data package inside the common module. From there, build a reference implementation in the exec/ref module.
        Hide
        Michael Hausenblas added a comment -

        OK, so I took that over. Though I think I know the current codebase I'd appreciate directions where and how to start this.

        Show
        Michael Hausenblas added a comment - OK, so I took that over. Though I think I know the current codebase I'd appreciate directions where and how to start this.
        Michael Hausenblas made changes -
        Field Original Value New Value
        Assignee Michael Hausenblas [ mhausenblas ]
        Julian Hyde created issue -

          People

          • Assignee:
            Jason Altekruse
            Reporter:
            Julian Hyde
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development