Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-2006

AsFactor has unexpected behavior when partitions not set

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 0.13.2
    • Algorithms
    • None
    • Zeppelin Notebook, Spark 2.1, scala 2.10

    Description

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)), numPartitions = 2)

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 =>

      {0:1.0}

      1 =>

      {0:1.0}

      2 =>

      {1:1.0}

      3 =>

      {0:1.0}

      4 => {}
      }
      ```

      as expected, however

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)))

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 => {}
      1 => {}
      2 => {}
      3 => {}
      4 => {}
      }
      ```

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            rawkintrevo Trevor Grant

            Dates

              Created:
              Updated:

              Slack

                Issue deployment