Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
0.9.0
-
None
-
None
Description
To reproduce:
Create a job that reads from two topics with equal number of partitions > 1. Configure the job to bootstrap from one of the topics and set yarn.container.count > 1.
Observed outcome:
You'll see that the container completes bootstrapping for the partitions assigned to it's tasks but never finishes bootstrapping for the remaining partitions that are assigned to other containers.
Expected outcome:
When each task as finished bootstrapping the partitions it's responsible for, bootstrapping should be considered complete.
Debug logs available here: https://mail-archives.apache.org/mod_mbox/samza-dev/201506.mbox/%3CCAPOm%3DTN88YSbn_pzpb-%2BwGxc619LsE%3D-XAfaydBKmLrTLOf4QA%40mail.gmail.com%3E