[SPARK-16440] Undeleted broadcast variables in Word2Vec causing OoM for long runs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0, 1.6.1, 1.6.2, 2.0.0
• Fix Version/s: 1.6.3, 2.0.1
• Component/s: MLlib
• Assignee: Anthony Truchet
• Reporter: Anthony Truchet
• Created: 08/Jul/16 12:45
• Updated: 28/Feb/17 10:18
• Resolved: 13/Jul/16 10:40

Description

Three broadcast variables created at the beginning of Word2Vec.fit() are never deleted nor unpersisted. This seems to cause excessive memory consumption on the driver for a job running hundreds of successive training.

They are

    val expTable = sc.broadcast(createExpTable())
    val bcVocab = sc.broadcast(vocab)
    val bcVocabHash = sc.broadcast(vocabHash)

Issue Links

• relates to: SPARK-11898 Use broadcast for the global tables in Word2Vec
• links to: [Github] Pull Request #14153 (srowen)
• links to: [Github] Pull Request #14268 (AnthonyTruchet)
• links to: [Github] Pull Request #14299 (AnthonyTruchet)