[BEAM-4199] [SQL] Add a DLQ support for Pubsub tables

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: dsl-sql
• Assignee: Anton Kedin
• Reporter: Anton Kedin
• Created: 30/Apr/18 21:22
• Updated: 24/Jul/20 20:17
• Resolved: 23/May/18 03:41

Description

Currently we crash the pipeline if there's any error processing the message from the pubsub, including if it has incorrect JSON format, like missing fields etc.

Correct solution would be for the user to specify a way to handle the errors, and ideally point to a dead-letter-queue where Beam should send the messages it could not process.

Issue Links

• links to: GitHub Pull Request #5380