[SAMZA-502] BootstrappingChooser never unblocks if bootstrap topic is initially empty

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: container
• Assignee: Yan Fang
• Reporter: Roger Hoover
• Created: 04/Jan/15 23:20
• Updated: 07/Jan/15 16:33
• Resolved: 07/Jan/15 16:00

Description

To reproduce:

• Create a job that bootstraps topic "config" and consumes topic "events"
• Create both topics in Kafka but only populate the "events" topic
• Start the job
• Now populate the "config" topic

What happens:

• The job will consume the messages in the "config" topic but not unblock the "events" topic

What should happen:

• Either
  • Samza should consider an empty bootstrap topic as "caught up" and immediately start processing the "event" topic
  • Samza should consider an empty bootstrap topic as a fatal error and exit

I uploaded some trace logs here in case they help: http://pastebin.com/RsdwYkRk
In this case, the bootstrap topic is called "ops.web.parse_rules__ck_4_module" and the event topic is called "ops._raw.web_parsed"

I'm testing using the ProcessJobFactory in case it matters.