[SAMZA-2087] Use separate thread pools for AsyncStreamTaskAdapter and AsyncRunLoop

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: container
• Assignee: Bharath Kumarasubramanian
• Reporter: Bharath Kumarasubramanian
• Created: 24/Jan/19 23:58
• Updated: 04/Apr/19 04:03

Description

Currently, AsyncStreamTaskAdapter and AsyncRunLoop share thread pools and the thread pool size is governed by job.container.thread.pool.size. This introduces disparity in the semantics of task.max.concurrency between async stream task vs adapted async stream task.

task.max.concurrency governs the parallelism within a task. In case of applications that directly implement AsyncStreamTask, this corresponds to maximum number of inflight messages for a task. However, in the case of applications that get adapted to AsyncStreamTask, the maximum number of inflight messages are bounded by the job.container.thread.pool.size. In fact, all the tasks within a container share the thread pool which results in even lesser parallelism within a task even though applications configure task.max.concurrency to a much greater number than job.container.thread.pool.size

 

 

 

Issue Links

• links to: GitHub Pull Request #895