[MADLIB-640] SVM Novelty: Result is incorrect class which is "OUT"

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.9.1
• Labels: severity_set
• Assignee: Rahul Iyer
• Reporter: Jiali Yao
• Created: 06/Sep/12 07:27
• Updated: 23/Jun/16 16:57
• Resolved: 23/Jun/16 16:57

Description

We use data sets which class = 1 as "IN" and verify result in two side:
1. All class = 1 items should be as marked as "IN"
2. All class = -1 items should be as marked as "OUT"

But in MADlib result, we can see that all items are marked as "IN".
Below is one example:

-- method: svm_nd_dot_ds_0_0_svm_novelty_detection_0
SELECT madlib.svm_novelty_detection
                        ( 'madlibtestdata.svm_a9a_in'::text     --input_table
                        , 'madlibtestresult.nd_model_table'::text    --model_table
                        , 'true'::boolean       --parallel
                        , 'madlib.svm_dot'::text    --kernel_func
                        , 'false'::boolean        --verbose
                        , '0.01'::float8            --eta
                        , '0.005'::float8             --nu
                   ) AS q;
-- All class should be return -1
-- method: svm_nd_dot_ds_0_0_svm_nd_predict_score_3
SELECT madlibtestdata.svm_nd_predict_score
                        ( 'madlibtestresult.nd_model_table'::text     --model_table
                        , 'madlibtestdata.svm_a9a_out'::text     --input_table
                        , 'true'::boolean        --parallel
                   ) AS score;

Currently for this query all class return 1.

Data sets name TrainSize TestSize Attributes Rate(1:-1) Missing Source URL
a9a 32561 16281 123 11687:37155 N http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html#a9a

We have tested multiple datasets,
Parallel = true:
5/5 dataset on Gaussian return score = 0
5/5 dataset on Polyminal return score = 0
4/5 dataset on dot return score = 0
Parallel = false:
5/5 dataset on Gaussian return score = 0
3/5 dataset on Polyminal return score = 0
1/5 dataset on dot return score = 0

When score > 0, we have one data set has better result than libsvm.
When in parallel = true, we use most vote model as select prediction value