[REEF-335] Remove the use of managed C++ in the bridge

• Type: Improvement
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Component/s: REEF Bridge
• Labels: CoreCLR
• Assignee: Doug Service
• Reporter: Markus Weimer
• Created: 19/May/15 17:31
• Updated: 23/May/18 20:40

Description

Some of the code in the bridge is written in managed C++. That isn't available as part of the open source .NET (yet). We should replace those code parts with C# or native C++.

Issue Links

• is blocked by: REEF-1567 Update Newtonsoft Json.NET to version 10.0.3
• is required by: REEF-334 Make sure REEF.NET builds and runs on Linux
• relates to: REEF-2023 New Java Bridge with gRPC based implementation.

Sub-Tasks

• Insert C# interface between managed C++ and current C#: Closed
• Replace managed C++ with pure C++: Closed
• Add avsc to C# code generation for the Java/Bridge interop to the C# build.: Resolved
• Move current bridge Avro messages to lang/common/bridge/avro: Resolved
• Replace interop call to StartHandlerOnNext() between Java/C# bridges: Closed
• Split the manged bridge into an executable C# assembly and Java jar file: Open
• Add protocol negotiation to the Java/C# avro interface so header message strings can be replace with integers.: Closed
• Avro bridge C# code generation: Resolved
• Add Avro message protocol serializer to C# side: Resolved
• Design and implement Avro Java/C# bridge as compile time option: Closed
• Add Avro message protocol serializer to Java side: Resolved
• Modify msbuild Avro code generation to support multiple CS from single avsc.: Resolved
• Utility class that converts an asynchronous API into a synchronous API: Resolved
• Replace interop call to clrSystemAllocatedEvaluatorHandlerOnNext() between Java/C# bridges: Closed
• Replace interop call to clrSystemActiveContextHandlerOnNext(): Open
• Replace interop call to clrSystemTaskMessageHandlerOnNext(): Open
• Replace interop call to clrSystemFailedTaskHandlerOnNext(): Open
• Replace interop call to clrSystemHttpServerHandlerOnNext(): Open
• Replace interop call to clrSystemFailedEvaluatorHandlerOnNext(): Open
• Replace interop call to clrSystemCompletedTaskHandlerOnNext(): Open
• Replace interop call to clrSystemRunningTaskHandlerOnNext(): Open
• Replace interop call to clrSystemSuspendedTaskHandlerOnNext(): Open
• Replace interop call to clrSystemCompletedEvaluatorHandlerOnNext(): Open
• Replace interop call to clrSystemClosedContextHandlerOnNext(): Open
• Replace interop call to clrSystemFailedContextHandlerOnNext(): Open
• Replace interop call to clrSystemContextMessageHandlerOnNext(): Open
• Replace interop call to callClrSystemOnRestartHandler(): Open
• Replace interop call to clrSystemDriverRestartActiveContextHandlerOnNext(): Open
• Replace interop call to clrSystemDriverRestartRunningTaskHandlerOnNext(): Open
• Replace interop call to clrSystemDriverRestartCompletedHandlerOnNext(): Open
• Replace interop call to clrSystemDriverRestartFailedEvaluatorHandlerOnNext(): Open
• Replace interop call to clrSystemProgressProviderGetProgress(): Open
• Replace interop call to clrSystemSetupBridgeHandlerManager(): Open
• Use external executor and Runnable in MultiAsyncToSync: Resolved
• Create RemoteManager with given host and port; clean up the RM factory API: Resolved
• Make ProtocolSerializer java class injectable by Tang: Resolved
• Use injection in C# ProtocolSerializer: Closed
• Implement network transport for the Java-CLR bridge: Closed
• Implement network transport for the Java-CLR bridge in C#: Resolved
• Message framework skeleton for all Java to C# calls: Closed
• Cross language tests of bridge messaging components: Closed
• Proof of concept replacing managed C++ with pinvoke: Closed
• Add native dll project for bridge pinvoke code: Closed
• Add new C# .NET Core executable project to dotnet solution for C# side of two process bridge: Resolved
• Add gRPC code generation to C# side of two process bridge project: Resolved
• Create Java project for gRPC two process bridge: Resolved
• Make Driver Service a static configuration: Open
• Implement C# bridge client using gRPC generated stubs: In Progress
• Implement HelloREEF on C# bridge client: In Progress
• Add Azure batch support to new bridge: Open