[APEXMALHAR-1634] Improve the BlockReader partitioning scheme to accommodate ingestion rate

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Fix Version/s: 2.1.0
• Component/s: adapters other
• Assignee: Chandni Singh
• Reporter: Chandni Singh
• Created: 17/Jan/15 19:37
• Updated: 26/Feb/15 00:52
• Resolved: 26/Feb/15 00:52

Description

Currently the dynamic partitioning scheme of block reader doesn't take ingestion rate into account.
The way it can be controlled by is setting a lower threshold on block readers and reducing the number of block readers.

We need a way to specify maximum ingestion rate which would limit the physical partitions that are created