Description
Inline tables were added in to Spark SQL in 2.0, e.g.: select * from values (1, 'A'), (2, 'B') as tbl(a, b)
This is currently implemented using a LocalRelation and this relation is created during parsing. This has several weaknesses: you can only use simple expressions in such a plan, and type coercion is based on the first row in the relation, and all subsequent values are cast in to this type. The latter violates the principle of least surprise.
I would like to rewrite this into a union of projects; each of these projects would contain a single table row. We apply better type coercion rules to a union, and we should be able to rewrite this into a local relation during optimization.
Attachments
Issue Links
- is related to
-
SPARK-17150 Support SQL generation for inline tables
-
- Resolved
-
- links to
