[SYSTEMDS-1973] Optimization of parameters, Hyperparameters, and testing.

• Type: Epic
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: SystemDS 2.1
• Component/s: Algorithms, APIs, Documentation, Test
• Epic Name: Optimization of parameters & Hyperparameters
• Assignee: Janardhan Pulivarthi
• Reporter: Janardhan Pulivarthi
• Created: 24/Oct/17 17:00
• Updated: 14/May/21 10:56
• Resolved: 14/May/21 10:56

Description

This Epic tracks the algorithm optimization related improvements, and their testing.

Phase 1: Addition of support for bayesian optimization.

This procedure constructs a probabilistic model for f(x), and then exploits this model to make decisions about where in input space to next evaluate the function, while integrating out uncertainity. The essential philosophy is to use all of the information available from previous evaluations of f(x).

 

When performing Bayesian Optimization, SYSTEMML-979

1. one must select a prior over functions that will express assumptions about the function being optimized. – We choose Gaussian Process Prior

2. need an acquisition function, which is used to construct a utility function from the model posterior, allowing us to determine the next point to evaluate.

 

Phase 2: Addition of Model selection & cross validation support at Engine level or API side.

Once the bayesian optimization is supported, the module is integrated into our API as described in SYSTEMML-1962 . By wrapping the dml functions in the optimization algorithms and invoking them either by java or python scripts.

 

Phase 3: Addition of Optimization test functions.

Testing of the training is done with the help of the well known benchmark functions, SYSTEMML-1974 , which can be imported or can be invoked with the help of python scripts or just by importing the function into the dml script at hand.

Issue Links

• contains: SYSTEMDS-979 Add support for bayesian optimization
• contains: SYSTEMDS-1962 Support model-selection via mllearn APIs