Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6650

ExecutorAllocationManager never stops

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.1, 1.4.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      ExecutorAllocationManager doesn't even have a stop() method. That means that when the owning SparkContext goes away, the internal thread it uses to schedule its activities remains alive.

      That means it constantly spams the logs and does who knows what else that could affect any future contexts that are allocated.

      It's particularly evil during unit tests, since it slows down everything else after the suite is run, leaving multiple threads behind.

        Attachments

          Activity

            People

            • Assignee:
              vanzin Marcelo Vanzin
              Reporter:
              vanzin Marcelo Vanzin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: