Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-390 High-Level Language for Samza
  3. SAMZA-483

A common representation of relational algebra for streaming SQL

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: sql
    • Labels:

      Description

      Per discussion with Chris Riccomini and Milinda Pathirage, we agreed that it seems to be a good idea to define a common representation of relational algebra on top of the operators defined in the operator layer (see SAMZA-482), which can be the common base that we can use to generate the description/configuration of a Samza job.

      This common layer can also be used by DSL-like language parser as a result of parsing a DSL program.

      Some additional requirements needed in addition to pure relational algebra:
      1) the common representation should include window operators and stream operators (i.e. IStream/DStream/RStream)
      2) the common representation should include description on parallelism of the jobs (i.e. how many partitions the resultant Samza job will use)

      Some references:
      http://web.cs.wpi.edu/~mukherab/i/DCAPE.pdf
      https://cs.uwaterloo.ca/~david/cs848/stream-cql.pdf
      http://davis.wpi.edu/dsrg/PROJECTS/CAPE/publications.htm
      http://davis.wpi.edu/dsrg/PROJECTS/CAPE/slides.htm

        Attachments

        1. calcite-integration-prototype.patch
          39 kB
          Milinda Pathirage
        2. SAMZA-483-1.patch
          169 kB
          Milinda Pathirage
        3. SAMZA-483-2.patch
          187 kB
          Milinda Pathirage
        4. SAMZA-483-3.patch
          33 kB
          Milinda Pathirage
        5. SAMZA-483-4.patch
          1 kB
          Milinda Pathirage
        6. SAMZA-483-5.patch
          34 kB
          Milinda Pathirage
        7. SAMZA-483-6.patch
          34 kB
          Milinda Pathirage
        8. SAMZA-483-7.patch
          34 kB
          Milinda Pathirage

          Issue Links

            Activity

              People

              • Assignee:
                milinda Milinda Pathirage
                Reporter:
                nickpan47 Yi Pan
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: