[REEF-1771] Enable IDistributedDataSet in .NET for Parquet files

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.16
• Component/s: REEF
• Assignee: Unassigned
• Reporter: Shouheng Yi
• Created: 11/Apr/17 18:33
• Updated: 12/Apr/17 23:27

Description

We want to enable REEF to ingest distributed parquet files and represent them as IDistributedDataSet in .Net. This allow us to better integrate with existing machine learning frameworks that heavily rely on Parquet format.

Initial step has been taken in REEF-1765. Future works will be sub items toward this goal.

Sub-Tasks

• Building a Parquet Reader for Potential Integrations with Other ML Frameworks: Closed
• Adding C# ParquetReader: Open
• Add hadoop-core for reef-experimental: Closed