[SPARK-13125] makes the ratio of KafkaRDD partition to kafka topic partition configurable.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.6.1
• Component/s: Input/Output
• Labels: features
• Flags: Patch, Important
• Assignee: Unassigned
• Reporter: zhengcanbin
• Shepherd: zhengcanbin
• Created: 02/Feb/16 03:48
• Updated: 10/Mar/16 00:50
• Resolved: 03/Feb/16 09:06

Description

Now each given Kafka topic/partition corresponds to an RDD partition, in some case it's quite necessary to make this configurable, namely a ratio configuration of RDDPartition/kafkaTopicPartition is needed.

Sub-Tasks

• add 'spark.streaming.kafka.partition.multiplier' into SparkConf: Closed