We are seeing a performance degradation for python streaming word count load tests.
After some investigation, it appears to be caused by swapping the original ThreadPoolExecutor to UnboundedThreadPoolExecutor in sdk worker. Suspicion is that python performance is worse with more threads on cpu-bounded tasks.
A simple test for comparing the multiple thread pool executor performance:
<apache_beam.utils.thread_pool_executor.UnboundedThreadPoolExecutor object at 0x7fab400dbe50> uses 268.160675049
<concurrent.futures.thread.ThreadPoolExecutor object at 0x7fab40096290> uses 79.904583931
<concurrent.futures.thread.ThreadPoolExecutor object at 0x7fab400dbe50> uses 191.179054976
1 Thread ThreadPoolExecutor:
12 Threads ThreadPoolExecutor: