Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4047

Generate runtime warning for naive implementation examples for algorithms implemented in MLlib/graphx

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: Examples
    • Labels:

      Description

      Based on SPARK-2434, we're generating runtime warnings to denote that the example implementation of algorithms were naive and a well implemented version is available in MLlib. Here are list of examples that are related to algorithms implemented in MLlib and graphx.

      1. LocalALS
      2. LocalFileLR
      3. LocalKMeans
      4. LocalLR
      5. SparkALS
      6. SparkHdfsLR
      7. SparkKMeans
      8. SparkLR
      9. SparkPageRank
      10. SparkTachyonHdfsLR

      Python examples:

      1. ALS
      2. kmeans
      3. logistic_regression
      4. pagerank

      Java examples:

      1. JavaHdfsLR
      2. JavaPageRank

      - Examples with missing runtime warnings.

      Also in few examples implementing LR, its currently pointing to org.apache.spark.mllib.classification.LogisticRegression instead of org.apache.spark.mllib.classification.LogisticRegressionModel

        Attachments

          Activity

            People

            • Assignee:
              srinathsmn Varadharajan
              Reporter:
              srinathsmn Varadharajan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified