Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
When working in S3, or even HDFS, parquet is pretty much the defacto binary format. It'd be great to be able to read and write to it.
At the same time it would be nice if you could specify size limits for parquet files (e.g. 128mb) so that we don't create one massive file.
If this could work via vfs then all the better! So you can read and write parquet directly from s3 etc.
