[BEAM-6261] Dataflow runner does not refresh updated side inputs

• Type: New Feature
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.9.0
• Component/s: runner-dataflow
• Assignee: Daniel Mills
• Reporter: Scott Wegner
• Created: 18/Dec/18 17:14
• Updated: 16/May/20 13:43
• Resolved: 29/Jan/20 07:27

Description

See this user@ thread.

The Slowly-changing lookup cache pattern described on the GCP blog uses a side input to feed lookup data to join with the main data input. However, the Dataflow runner doesn't update side inputs.

Example pipeline here: https://github.com/lbordwell/sideinput