[SAMZA-833] ProcessJob mishandling containers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1
• Assignee: Tao Feng
• Reporter: Jake Maes
• Created: 08/Dec/15 00:55
• Updated: 15/Apr/16 06:29
• Resolved: 15/Apr/16 06:29

Description

As a result of SAMZA-465 and SAMZA-805, ProcessJobFactory now passes the full config to the ProcessJob and no longer forces the container count to 1. This causes the ProcessJob to actually read the container count config and if it is not 1, it produces some unexpected behavior.

Specifically we've had reports of ProcessJobs dropping messages because the container count is > 1, so the grouper assigns partitions to more than 1 container, but only one container actually runs.

The goal of this ticket is to either force the container count to 1 for ProcessJob, or fix how multiple containers run with ProcessJob. But we should not allow the scenario where messages are dropped.