[SAMZA-483] A common representation of relational algebra for streaming SQL

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Component/s: sql
• Labels: project
• Assignee: Milinda Pathirage
• Reporter: Yi Pan
• Created: 26/Nov/14 18:58
• Updated: 27/Feb/15 21:26
• Resolved: 13/Feb/15 21:34

Description

Per discussion with criccomini and milinda, we agreed that it seems to be a good idea to define a common representation of relational algebra on top of the operators defined in the operator layer (see SAMZA-482), which can be the common base that we can use to generate the description/configuration of a Samza job.

This common layer can also be used by DSL-like language parser as a result of parsing a DSL program.

Some additional requirements needed in addition to pure relational algebra:
1) the common representation should include window operators and stream operators (i.e. IStream/DStream/RStream)
2) the common representation should include description on parallelism of the jobs (i.e. how many partitions the resultant Samza job will use)

Some references:
http://web.cs.wpi.edu/~mukherab/i/DCAPE.pdf
https://cs.uwaterloo.ca/~david/cs848/stream-cql.pdf
http://davis.wpi.edu/dsrg/PROJECTS/CAPE/publications.htm
http://davis.wpi.edu/dsrg/PROJECTS/CAPE/slides.htm

Issue Links

• relates to: CALCITE-605 Reduce dependency on third-party maven repositories