Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2087

Use separate thread pools for AsyncStreamTaskAdapter and AsyncRunLoop

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: container
    • Labels:
      None

      Description

      Currently, AsyncStreamTaskAdapter and AsyncRunLoop share thread pools and the thread pool size is governed by job.container.thread.pool.size. This introduces disparity in the semantics of task.max.concurrency between async stream task vs adapted async stream task.

      task.max.concurrency governs the parallelism within a task. In case of applications that directly implement AsyncStreamTask, this corresponds to maximum number of inflight messages for a task. However, in the case of applications that get adapted to AsyncStreamTask, the maximum number of inflight messages are bounded by the job.container.thread.pool.size. In fact, all the tasks within a container share the thread pool which results in even lesser parallelism within a task even though applications configure task.max.concurrency to a much greater number than job.container.thread.pool.size

       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bharathkk Bharath Kumarasubramanian
                Reporter:
                bharathkk Bharath Kumarasubramanian
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h