Add InputKinds LazyMap and LazyMultiMap that allow map lookup without reading everything to memory. They will be accessed through functions such as:
func(K) func(*V) bool (a keyed function that returns an iterator)
func(K) V (a keyed function that returns a slice of values)
On the execution layer, the new forms would need to be added to exec/sideinput.go
The inputs layer, for the actual abstraction using reflection:
The funcx package would need to be updated to detect the new parameter forms
as well has the DoFn graph validation code
They would need to be correctly translated into the pipeline protos:
and finally back to the newly created handlers in the exec package.
If implemented pre-generics, the code generator frontend, and backend would need to be updated to detect and generate code for efficient no-reflection overhead map access functions. https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/util/shimx/generate.go
Unit must be added throughout and Integration tests should be added to verify the functionality against portable beam runners.
And of course, the user GoDoc should be updated for the support.
See this lengthy email response for a more indepth guide to how Side Inputs operate. https://lists.apache.org/thread.html/ra42dc7ee30842f11740eff33f0afcd63702695878e427127e1268381%40%3Cdev.beam.apache.org%3E