[SYSTEMDS-1695] Create design/investigation document for parameter server

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.1
• Assignee: Janardhan Pulivarthi
• Reporter: Niketan Pansare
• Created: 14/Jun/17 22:39
• Updated: 30/Jan/18 20:32
• Resolved: 30/Jan/18 20:32

Description

Here are relevant links:

• https://issues.apache.org/jira/browse/SPARK-4590
• https://issues.apache.org/jira/browse/SPARK-6932
• https://docs.google.com/document/d/1SX3nkmF41wFXAAIr9BgqvrHSS5mW362fJ7roBXJm06o/edit

This document need not cover how to integrate a parameter server in SystemML (which comes later), but should discuss following questions:
1. Should we integrate existing parameter server implementation or build one from scratch ? What are the pros and cons of each approaches ? For example: https://github.com/rjagerman/glint
2. If we plan to integrate existing parameter server implementation, what is the interface ? (for exampe: update(matrix, new matrix) or update(matrix, delta)).
3. Can parameter server help in ML algorithms ? Some experiments might be helpful.