Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35271

After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or launch new spark jobs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.0.1, 3.1.1
    • None
    • ML, PySpark
    • None

    Description

      CrossValidator/TrainValidationSplit fit will spawn many threads to run trial tasks.
      if one of the trial tasks failed, the CrossValidator/TrainValidationSplit fit will raise error and break the main thread, but other backgroud threads running other trial tasks will continue to run, and trial tasks which are pending to run in thread queue will also continue to launch.

      Attachments

        Activity

          People

            weichenxu123 Weichen Xu
            weichenxu123 Weichen Xu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: