[SYSTEMDS-979] Add support for bayesian optimization

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: SystemML 1.1, SystemDS 2.1
• Component/s: Algorithms
• Labels: optimization
• Assignee: Janardhan Pulivarthi
• Reporter: Niketan Pansare
• Created: 29/Sep/16 18:41
• Updated: 14/May/21 10:55
• Resolved: 14/May/21 10:55

Description

Main paper: https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf

supplements:

• selecting the next point to evaluate Jasper
• sobol sequence generator P Bratley, BL Fox
• Handling Sparsity via the Horseshoe Carlos M. Carvalho

Design document: http://www.bit.do/systemml-bayesian

Issue Links

• Is contained by: SYSTEMDS-1973 Optimization of parameters, Hyperparameters, and testing.
• relates to: SYSTEMDS-1159 Enable Remote Hyperparameter Tuning

Sub-Tasks

• Add XOR built in operator: Resolved
• Implemenation of Mode finding for Gaussian Process: Resolved
• Implementation of Sobol quasi random sequence generator: In Progress
• Implementation of Gaussian Process Classification: Resolved
• Implementation of Gaussian Process Regression: In Progress
• Implementation of Surrogate data slice sampling: In Progress
• Implementation of Bayesian module: In Progress
• implement acquistion functions for different scenarios: Open
• Implement covariance kernels: Resolved
• Addition of parallel monte carlo acquisition function: Open