[BEAM-12499] Processing message in sequence in a pipeline from IBM MQ (deliver message in order)

• Type: Wish
• Status: Triage Needed
• Priority: P3
• Resolution: Unresolved
• Component/s: beam-model
• Assignee: Unassigned
• Reporter: Aswini Kumar Das
• Created: 16/Jun/21 06:36
• Updated: 04/Jun/22 20:51

Description

We want to create a data pipeline that will read message from IBM MQ. And process the messages in the order it receive the message from IBM MQ. IBM MQ will deliver the message in order. The message does not have any intelligence (for example a timestamp or sequence number)..Would appreciate if it possible to achieve it without using single threaded model.

We want to scale it we would like to have ordering & parallelism, not sure whether its doable or not. Request your help. The runner is google dataflow.