[MXNET-731] Memory Management in Scala binding

• Type: Story
• Status: To Do
• Priority: Major
• Resolution: Unresolved
• Component/s: Apache MXNet Scala API
• Sprint: Scala/Java 1 (07/30-08/13), Scala/Java 2 (08/13-08/27), Scala/Java 4 (09/10-09/24), Scala/Java 5 (09/24-10/08), Scala/Java 6 (10/08 - 10/22), Scala/Java 7 (10/22-11/5), Scala/Java 8 (11/5 - 11/19), Scala/Java 9 (11/19-12/3), Scala/Java 10 (12/3-12/17), Scala/Java 13 (01/14 - 01/28)
• Epic Link: Scala API improvements
• Assignee: Naveen Swamy
• Reporter: Denis Davydenko
• Created: 26/Jul/18 01:23
• Updated: 28/Jan/19 20:43

Description

Due to C++ nature of MXNet, Scala binding has to take care about memory management and garbage collector for its users. Success criteria: Scala API users should not have to take special care about disposing objects created by MXNet and hence should not see memory leaks with their implementation of MXNet using Scala API binding.

Sub-Tasks

• Research how ND4J uses Garbage collector to manage off-heap memory: Done
• Design & Prototype of Memory Management of Off-Heap objects created in Scala: Done
• Research using Finalizers that runs inside a Single thread: Done
• Conduct discussion on dev@ list for Scala API memory management proposal: Done
• Write design document and publish to cwiki: Done
• Create generic NativeResource: Done
• Create ResNet50 model to use for StressTesting and PerformanceTest: Done
• Update Module.Scala to resolve Memory Leaks: In Progress
• Make ResourceScope work in Java.: Done
• Refactor and Fix Memory Leak in FeedForward.scala: To Do