Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-2006

AsFactor has unexpected behavior when partitions not set

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • 0.13.2
    • None
    • None
    • Zeppelin Notebook, Spark 2.1, scala 2.10

    Description

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)), numPartitions = 2)

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 =>

      {0:1.0}

      1 =>

      {0:1.0}

      2 =>

      {1:1.0}

      3 =>

      {0:1.0}

      4 => {}
      }
      ```

      as expected, however

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)))

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 => {}
      1 => {}
      2 => {}
      3 => {}
      4 => {}
      }
      ```

      Attachments

        Activity

          People

            Unassigned Unassigned
            rawkintrevo Trevor Grant
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: