[MADLIB-1335] Add new param metrics_compute_frequency to madlib_keras_fit()

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.16
• Component/s: Deep Learning
• Assignee: Unassigned
• Reporter: Ekta Khanna
• Created: 01/May/19 21:33
• Updated: 06/Jun/19 00:36
• Resolved: 06/Jun/19 00:36

Description

JIRA: https://issues.apache.org/jira/browse/MADLIB-1335

Context

Getting per iteration loss and other metrics for training and validation data sets can be expensive. This parameter is intended to give control to user on how often to do this computation.

Story

As a data scientist, I want to specify how often to calculate loss and other metrics for training and validation data (if provided), i.e, every n iterations

Interface

madlib_keras_fit(
    source_table            VARCHAR,
    model                   VARCHAR,
    dependent_varname       VARCHAR,
    independent_varname     VARCHAR,
    model_arch_table        VARCHAR,
    model_arch_id           INTEGER,
    compile_params          VARCHAR,
    fit_params              VARCHAR,
    num_iterations          INTEGER,
    use_gpu                 BOOLEAN,
    validation_table        VARCHAR,
    metrics_compute_frequency      INTEGER   <-------- NEW OPTIONAL PARAM
    name                    VARCHAR,
    description             VARCHAR

where `metrics_compute_frequency` is an optional parameter that means:

NULL  - calculate loss and metrics only on final model after last iteration (default)
n - calculate loss metrics every n-th iteration and on final model after last iteration

and `metrics_compute_frequency must be >=1 and <=num_iterations`

Acceptance

1) Set `num_iterations = 12` and leave ` metrics_compute_frequency` as default and get 1 loss and metrics value at end after 12 iterations
2) Set `num_iterations = 12` and set `metrics_compute_frequency = 5` and get loss and metrics after 5th and 10th iterations and at end after 12 iterations
3) Set `metrics_compute_frequency = 0` and get an error
4) Set `metrics_compute_frequency = num_iterations+1` and get an error
5) Test with different metrics: from https://keras.io/metrics/ try 'mae' , 'acc' etc.

Reference

[1] https://keras.io/metrics/