Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-2006

AsFactor has unexpected behavior when partitions not set

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.13.2
    • Component/s: Algorithms
    • Labels:
      None
    • Environment:

      Zeppelin Notebook, Spark 2.1, scala 2.10

      Description

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)), numPartitions = 2)

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 =>

      {0:1.0}

      1 =>

      {0:1.0}

      2 =>

      {1:1.0}

      3 =>

      {0:1.0}

      4 => {}
      }
      ```

      as expected, however

      ```
      val drmA = drmParallelize(dense((0.0), (0.0), (1.0), (0.0), (2.0)))

      val factorizer = new AsFactor().fit(drmA)

      val factoredA = factorizer.transform(drmA).collect
      ```

      Yields:
      ```
      drmA: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@75dcf2b2
      factorizer: org.apache.mahout.math.algorithms.preprocessing.AsFactorModel = org.apache.mahout.math.algorithms.preprocessing.AsFactorModel@13b49f81
      factoredA: org.apache.mahout.math.Matrix =
      {
      0 => {}
      1 => {}
      2 => {}
      3 => {}
      4 => {}
      }
      ```

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            rawkintrevo Trevor Grant
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development