[FLINK-4257] Handle delegating algorithm change of class

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Component/s: Library / Graph Processing
• Assignee: Greg Hogan
• Reporter: Greg Hogan
• Created: 22/Jul/16 17:27
• Updated: 08/Sep/16 21:15
• Resolved: 08/Sep/16 21:15

Description

A class created by ProxyFactory can intercept and reinterpret method calls using its MethodHandler, but is restricted in that

• the type of the proxy class cannot be changed
• method return types must be honored

We have algorithms such as VertexDegree and TriangleListing that change return type depending on configuration, even between single and dual input functions. This can be problematic, e.g. in OperatorTranslation where we test dataSet instanceof SingleInputOperator or dataSet instanceof TwoInputOperator.

Even simply changing operator can be problematic, e.g. MapOperator.translateToDataFlow returns MapOperatorBase whereas ReduceOperator.translateToDataFlow returns SingleInputOperator.

I see two ways to solve these issues. By adding a simple NoOpOperator that is skipped over during OperatorTranslation we could wrap all algorithm output and always be proxying the same class.

Alternatively, making changes only within Gelly we can append a "no-op" pass-through MapFunction to any algorithm output which is not a SingleInputOperator. And Delegate can also walk the superclass hierarchy such we are always proxying SingleInputOperator.

There is one additional issue. When we call DataSet.output the delegate's MethodHandler must reinterpret this call to add itself to the list of sinks.

Issue Links

• depends upon: FLINK-4265 Add a NoOpOperator
• links to: GitHub Pull Request #2474