Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-477

SimilarityMatrixEntryKeyPartitioner sometimes produces illegal partition numbers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.4
    • None

    Description

      My job failed on EMR today with the exception below. I think this happens only when when there are negative row/column indices in a matrix given to RowSimilarityJob, still have to figure out the details.

      java.io.IOException: Illegal partition for org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey@3532b21 (-3)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:921)
      at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:549)
      at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
      at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:629)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:310)
      at org.apache.hadoop.mapred.Child.main(Child.java:170)

      Attachments

        1. MAHOUT-477.patch
          2 kB
          Sebastian Schelter

        Activity

          People

            srowen Sean R. Owen
            ssc Sebastian Schelter
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: