Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5108 Changes needed for Binary Compatibility for MR applications via YARN
  3. MAPREDUCE-5530

Binary and source incompatibility in mapred.lib.CombineFileInputFormat between branch-1 and branch-2

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.1.1-beta
    • 2.2.0
    • mrv1, mrv2
    • None

    Description

      mapred.lib.CombineFileInputFormat in branch-1 has this method:

      protected boolean isSplitable(FileSystem fs, Path file)
      

      In branch-2, mapred.lib.CombineFileInputFormat is now a subclass of mapreduce.lib.input.CombineFileInputFormat, from which it inherits the similar method:

      protected boolean isSplitable(JobContext context, Path file)
      

      This means that any code that subclasses mapred.lib.CombineFileInputFormat and does not provide its own implementation of protected boolean isSplitable(FileSystem fs, Path file) will not be binary or source compatible if it tries to call isSplitable with a FileSystem argument anywhere (that is, if compiled against branch-1, it will throw a NoSuchMethodError if run against branch-2; also, it won't even compile against branch-2).

      Attachments

        1. MAPREDUCE-5530.patch
          2 kB
          Robert Kanter
        2. MAPREDUCE-5530.patch
          2 kB
          Robert Kanter
        3. MAPREDUCE-5530.patch
          3 kB
          Robert Kanter
        4. MAPREDUCE-5530.patch
          3 kB
          Robert Kanter

        Activity

          People

            rkanter Robert Kanter
            rkanter Robert Kanter
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: