Mahout
  1. Mahout
  2. MAHOUT-614

org.apache.mahout.classifier.baytes.MultipleOutputFormat not working as intended with Hadoop 0.20?

    Details

      Description

      I believe there might be an error in org.apache.mahout.classifier.baytes.MultipleOutputFormat. It overrides the Hadoop class FileOutputFormat, and most of its work is done in getRecordWriter(FileSystem, Configuration, String, Progressable). However this is not the method that one must override to control how FileOutputFormat writes records; that's getRecordWriter(TaskAttemptContext). My hunch is that this used to work, but against the Hadoop 0.19.x APIs. (@Override is our friend!)

      I've attached a patch that I believe addresses this and along the way is able to clean things up slightly. Am I on track here?

        Activity

        Sean Owen created issue -
        Sean Owen made changes -
        Field Original Value New Value
        Attachment MAHOUT-614.patch [ 12471554 ]
        Hide
        Robin Anil added a comment -

        I will take a look from home. Patch seems alright, just have to verify the functionality.

        Show
        Robin Anil added a comment - I will take a look from home. Patch seems alright, just have to verify the functionality.
        Hide
        Robin Anil added a comment -

        Verified using 20newsgroups. Looks good to commit.

        Show
        Robin Anil added a comment - Verified using 20newsgroups. Looks good to commit.
        Sean Owen made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Mahout-Quality #640 (See https://hudson.apache.org/hudson/job/Mahout-Quality/640/)
        MAHOUT-614 fix up overriding of Hadoop's FileOutputFormat

        Show
        Hudson added a comment - Integrated in Mahout-Quality #640 (See https://hudson.apache.org/hudson/job/Mahout-Quality/640/ ) MAHOUT-614 fix up overriding of Hadoop's FileOutputFormat
        Sean Owen made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Robin Anil
            Reporter:
            Sean Owen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development