Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3697

Hadoop Counters API limits Oozie's working across different hadoop versions

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.1
    • 0.23.1
    • mrv2
    • None

    Description

      Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
      hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
      parent class AbstractCounters implements it. This is different from hadoop20X.
      As a result, Oozie compiled with either hadoop version does not work with the
      other version.
      A specific scenario, Oozie compiled with .23 and run against 205, does not
      update job status owing to a Counters API exception.

      Will explicit re-compilation against the relevant hadoop jars be required each
      time? This will prevent launching a uniform Oozie version across different
      clusters.

      Attachments

        1. apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
          1 kB
          Thomas White
        2. MAPREDUCE-3697.patch
          31 kB
          Mahadev Konar
        3. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        4. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        5. MAPREDUCE-3697.patch
          23 kB
          Mahadev Konar
        6. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar
        7. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar
        8. MAPREDUCE-3697-facade.patch
          6 kB
          Alejandro Abdelnur
        9. MAPREDUCE-3697-facadeV2.patch
          6 kB
          Alejandro Abdelnur
        10. MAPREDUCE-3697-facadeV3.patch
          10 kB
          Alejandro Abdelnur

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mahadev Mahadev Konar
            johnvijoe John George
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment