Details
-
New Feature
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
Description
We use BeamSQL to run streaming jobs mostly, and add a join_as_lookup improvement(internal branch) to cover the streaming-to-batch case(similar as [1]). I could submit a PR as experimental if people are interested.
The rough solution is, if one source of join node implements BeamSeekableTable and the other is not, then the join node is converted to a fact-lookup operation.
Ref:
[1] https://docs.google.com/document/d/1B-XnUwXh64lbswRieckU0BxtygSV58hysqZbpZmk03A/edit?usp=sharing
xumingming takidau for any comments
Attachments
Issue Links
- links to