[SPOT-185] [Ingest] New Ingest Framework v2

• Type: Epic
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0
• Epic Name: New Ingest Framework based on Scala and Spark
• Assignee: Everardo Lopez Sandoval
• Reporter: Everardo Lopez Sandoval
• Created: 23/Jun/17 18:03
• Updated: 29/Jun/17 18:07

Description

A new Ingest framework is required to support new pipelines and new pipeline sources:

• Scala/Spark based.
• Use dynamic mappers to map raw data to Spot schema.
• Support Batch and Streaming Use case.