Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3697

Hadoop Counters API limits Oozie's working across different hadoop versions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.1
    • 0.23.1
    • mrv2
    • None

    Description

      Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
      hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
      parent class AbstractCounters implements it. This is different from hadoop20X.
      As a result, Oozie compiled with either hadoop version does not work with the
      other version.
      A specific scenario, Oozie compiled with .23 and run against 205, does not
      update job status owing to a Counters API exception.

      Will explicit re-compilation against the relevant hadoop jars be required each
      time? This will prevent launching a uniform Oozie version across different
      clusters.

      Attachments

        1. MAPREDUCE-3697.patch
          31 kB
          Mahadev Konar
        2. apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
          1 kB
          Thomas White
        3. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        4. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        5. MAPREDUCE-3697.patch
          23 kB
          Mahadev Konar
        6. MAPREDUCE-3697-facadeV3.patch
          10 kB
          Alejandro Abdelnur
        7. MAPREDUCE-3697-facadeV2.patch
          6 kB
          Alejandro Abdelnur
        8. MAPREDUCE-3697-facade.patch
          6 kB
          Alejandro Abdelnur
        9. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar
        10. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar

        Issue Links

          Activity

            People

              mahadev Mahadev Konar
              johnvijoe John George
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: