[BEAM-301] Add a Beam SQL DSL

• Type: New Feature
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: dsl-sql, sdk-ideas
• Assignee: Mingmin Xu
• Reporter: Jean-Baptiste Onofré
• Created: 23/May/16 07:37
• Updated: 24/Jul/20 20:56
• Resolved: 11/Sep/17 20:49

Description

The SQL DSL helps developers to build a Beam pipeline from SQL statement in String directly.
In Phase I, it starts to support INSERT/SELECT queries with FILTERs, one example SQL as below:

INSERT INTO `SUB_USEREVENT` (`SITEID`, `PAGEID`, `PAGENAME`, `EVENTTIMESTAMP`)
(SELECT STREAM `USEREVENT`.`SITEID`, `USEREVENT`.`PAGEID`, `USEREVENT`.`PAGENAME`, `USEREVENT`.`EVENTTIMESTAMP`
FROM `USEREVENT` AS `USEREVENT`
WHERE `USEREVENT`.`SITEID` > 10)

A design doc is available at https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_0a1Bz5BsCROMzCU/edit?usp=sharing.

Issue Links

• is blocked by: BEAM-2833 TestPipeline fail to get Coder
• is related to: BEAM-1900 create feature branch 'dsls_sql' for Beam SQL
• links to: GitHub Pull Request #2479
• links to: GitHub Pull Request #2530
• links to: GitHub Pull Request #3606
• links to: GitHub Pull Request #3782
• links to: GitHub Pull Request #3794
• links to: Road Map for Beam SQL