Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1178

MultipleInputs fails with ClassCastException

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When running MultipleInputs against the new API, we get failures with this ClassCastException:

      java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.TaggedInputSplit cannot be cast to org.apache.hadoop.mapreduce.lib.input.FileSplit
      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:70)
      at org.apache.hadoop.mapreduce.lib.input.KeyValueLineRecordReader.initialize(KeyValueLineRecordReader.java:59)
      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:439)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:599)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:257)

      The unit test for MultipleInputs doesn't actually run a job so this snuck through while still passing the unit test. Attached patch fixes the unit test to expose the failure and does a little casting kung-fu in LineRecordReader to avoid the error.

        Attachments

        1. MAPREDUCE-1178.patch
          8 kB
          Jay Booth
        2. patch-1178.txt
          11 kB
          Amareshwari Sriramadasu
        3. patch-1178-1.txt
          11 kB
          Amareshwari Sriramadasu

          Activity

            People

            • Assignee:
              amareshwari Amareshwari Sriramadasu
              Reporter:
              jbooth Jay Booth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: