Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19071

Optimizations for ML Pipeline Tuning

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • ML

    Description

      This is a parent task to plan the addition of optimizations in ML tuning for parallel model evaluation and more efficiency with pipelines. They will benefit Crossvalidator and TrainValidationSplit when performing a parameter grid search. The proposal can be broken into 3 steps in order of simplicity:

      1. Add ability to evaluate models in parallel.

      2. Optimize param grid for pipelines, as described in SPARK-5844

      3. Add parallel model evaluation to the optimized pipelines from step 2

      See the linked design document for details on the proposed implementation.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bryanc Bryan Cutler
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: