[SPARK-11198] Support record de-aggregation in KinesisReceiver

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: DStreams
• Target Version/s: 1.6.0
• Assignee: Burak Yavuz
• Reporter: Xiangrui Meng
• Shepherd: Tathagata Das
• Created: 20/Oct/15 00:44
• Updated: 10/Nov/15 01:19
• Resolved: 10/Nov/15 01:19

Description

We need to check/implement the support for record de-aggregation and subsequence number. This is the documentation from AWS:

http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-kpl-consumer-deaggregation.html

Issue Links

• depends upon: SPARK-11127 Upgrade Kinesis Client Library to the latest stable version
• links to: [Github] Pull Request #9403 (brkyvz)