[MADLIB-907] Prediction Metrics

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.9.1
• Component/s: Module: Utilities
• Assignee: Orhan Kislal
• Reporter: Frank McQuillan
• Created: 02/Nov/15 22:32
• Updated: 25/Jul/16 19:35
• Resolved: 14/Jun/16 18:30

Description

Story

As a data scientist, I want to compute prediction metrics on my data, so that I can gauge model accuracy based on predicted values vs. actual values.

1) The PDL Tools modules "Prediction Metrics" [1] is an example of what could be ported to MADlib. Source code is located at [2].

2) Here is functionality from PDL tools to use as a starting point:

mf_mae
Mean Absolute Error.

mf_mape
Mean Absolute Percentage Error.

mf_mpe
Mean Percentage Error.

mf_rmse
Root Mean Square Error.

mf_r2
R-squared.

mf_adjusted_r2
Adjusted R-squared.

mf_binary_classifier
Metrics for binary classification.

mf_auc
Area under the ROC curve (in binary classification).

mf_confusion_matrix
Confusion matrix for a multi-class classifier.

References

[1] PDL Tools Prediction Metrics module
http://pivotalsoftware.github.io/PDLTools/group__grp__prediction__metrics.html
[2] PDL tools source code
https://github.com/pivotalsoftware/PDLTools

Issue Links

• links to: GitHub Pull Request #42