Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1178

MultipleInputs fails with ClassCastException

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • None
    • None
    • Reviewed

    Description

      When running MultipleInputs against the new API, we get failures with this ClassCastException:

      java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.TaggedInputSplit cannot be cast to org.apache.hadoop.mapreduce.lib.input.FileSplit
      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:70)
      at org.apache.hadoop.mapreduce.lib.input.KeyValueLineRecordReader.initialize(KeyValueLineRecordReader.java:59)
      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:439)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:599)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:257)

      The unit test for MultipleInputs doesn't actually run a job so this snuck through while still passing the unit test. Attached patch fixes the unit test to expose the failure and does a little casting kung-fu in LineRecordReader to avoid the error.

      Attachments

        1. MAPREDUCE-1178.patch
          8 kB
          Jay Booth
        2. patch-1178.txt
          11 kB
          Amareshwari Sriramadasu
        3. patch-1178-1.txt
          11 kB
          Amareshwari Sriramadasu

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amareshwari Amareshwari Sriramadasu Assign to me
            jbooth Jay Booth
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment