Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6617

Word2Vec is nondeterministic

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Auto Closed
    • 1.3.0
    • None
    • MLlib

    Description

      Word2Vec uses repartition: https://github.com/apache/spark/blob/v1.3.0/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala#L291, which doesn't provide deterministic ordering. This makes QA a little harder.

      Attachments

        Issue Links

          relates to

          Documentation - Documentation or Website SPARK-3356 Document when RDD elements' ordering within partitions is nondeterministic

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #5300 [Github] Pull Request #5300 (ilganeli)

          Activity

            People

              Unassigned Unassigned
              mengxr Xiangrui Meng
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: