Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1178

MultipleInputs fails with ClassCastException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When running MultipleInputs against the new API, we get failures with this ClassCastException:

      java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.TaggedInputSplit cannot be cast to org.apache.hadoop.mapreduce.lib.input.FileSplit
      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:70)
      at org.apache.hadoop.mapreduce.lib.input.KeyValueLineRecordReader.initialize(KeyValueLineRecordReader.java:59)
      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:439)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:599)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:257)

      The unit test for MultipleInputs doesn't actually run a job so this snuck through while still passing the unit test. Attached patch fixes the unit test to expose the failure and does a little casting kung-fu in LineRecordReader to avoid the error.

      1. patch-1178-1.txt
        11 kB
        Amareshwari Sriramadasu
      2. patch-1178.txt
        11 kB
        Amareshwari Sriramadasu
      3. MAPREDUCE-1178.patch
        8 kB
        Jay Booth

        Activity

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Jay Booth
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development