Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
A possibility of using PXF outside of its original HAWQ use case is pretty exciting. Calcite is also pretty exciting. This JIRA tracks my efforts of making the prototype of the two working together.
The first goal here could be make PXF available to Apache Calcite
as a backend (since Calcite itself doesn't deal with storage of data,
algorithms to process data, and a repository for storing metadata).
Calcite comes with a demo that allows you to treat a directory
full of CSV files as a DB (with individual files being tables) and
I wanted to extend that demo to use PXF reading CSV files from HDFS
instead:
http://calcite.apache.org/docs/tutorial.html
https://github.com/apache/calcite/tree/master/example/csv/src/main/java/org/apache/calcite/adapter/csv
Once that is done – we can grow from there and use Calcite to validate assumptions that exist between HAWQ and PXF