[SPARK-6514] For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as the Kinesis stream itself

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: DStreams
• Target Version/s: 1.4.0
• Assignee: Chris Fregly
• Reporter: Chris Fregly
• Created: 24/Mar/15 22:44
• Updated: 17/May/15 23:52
• Resolved: 17/May/15 23:52

Description

context: i started the original Kinesis impl with KCL 1.0 (not supported), then finished on KCL 1.1 (supported) without realizing that it's supported.

also, we should upgrade to the latest Kinesis Client Library (KCL) which is currently v1.2 right now, i believe.

Issue Links

• is related to: SPARK-6599 Improve usability and reliability of Kinesis stream
• links to: [Github] Pull Request #5882 (cfregly)
• links to: [Github] Pull Request #6147 (tdas)