[SAMOA-47] Integrate Avro Streams with SAMOA

• Type: New Feature
• Status: Done
• Priority: Minor
• Resolution: Done
• Component/s: SAMOA-API, SAMOA-Instances
• Labels: patch
• Flags: Patch
• Assignee: Unassigned
• Reporter: jayadeepj
• Created: 25/Oct/15 11:36
• Updated: 31/Jan/16 12:32
• Resolved: 30/Nov/15 11:27

Description

The current SAMOA readers can only support data streams in ARFF format. Hence SAMOA as a distributed streaming machine learning framework is limited in scope since end users may have to transform their data to ARFF . Apache Avro is a data serialization system that handles data streams in compact binary format and is typically used in conjunction with with Big Data eco-system tools. Avro allows two encodings for the data: Binary & JSON. Hence an Avro support may allow users with JSON data also to use SAMOA seamlessly.

The GOAL is to build support for Avro Streams into SAMOA by adding Avro File Stream Handler, Avro Loader to read records & transform to instances and a user option to switch between JSON/Binary encodings. The input format with representation of meta-data for both JSON/Binary data to be finalized along with build.