Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11302

Multivariate Gaussian Model with Covariance matrix returns incorrect answer in some cases

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.3.1, 1.4.1, 1.5.1, 1.6.0
    • Fix Version/s: 1.3.2, 1.4.2, 1.5.2, 1.6.0
    • Component/s: MLlib
    • Labels:
      None

      Description

      I have been trying to apply an Anomaly Detection model using Spark MLib.

      As an input, I feed the model with a mean vector and a Covariance matrix. ,assuming my features contain Co-variance.

      Here are my input for the model ,and the model returns zero for each data point for this input.

      MU vector -
      1054.8, 1069.8, 1.3 ,1040.1
      Cov' matrix -
      165496.0 , 167996.0, 11.0 , 163037.0
      167996.0, 170631.0, 19.0, 165405.0
      11.0, 19.0 , 0.0, 2.0
      163037.0, 165405.0 2.0 , 160707.0

      Conversely, for the non covariance case, represented by this matrix ,the model is working and returns results as expected
      165496.0, 0.0 , 0.0, 0.0
      0.0, 170631.0, 0.0, 0.0
      0.0 , 0.0 , 0.8, 0.0
      0.0 , 0.0, 0.0, 160594.2

        Attachments

          Activity

            People

            • Assignee:
              srowen Sean R. Owen
              Reporter:
              eyalsharon eyal sharon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: