Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3913

Pig should use job's jobClient wherever possible (fixes local mode counters)

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.13.0
    • 0.13.0
    • None
    • None

    Description

      MapreduceLauncher initializes a statsJobClient to poll counter information of jobs. This works fine in mapreduce mode but it reports incorrect information in local (auto-local) mode. Pig code should try to use org.apache.hadoop.mapred.jobcontrol.Job's getJobClient api to get handle to jobClient wherever possible. statsJobClient (and wherever its references are passed) should be deprecated.

      Attachments

        1. PIG-3913-1.patch
          8 kB
          Aniket Namadeo Mokashi
        2. PIG-3913-MRv2.v01.patch
          27 kB
          Gera Shegalov
        3. PIG-3913-3.patch
          29 kB
          Aniket Namadeo Mokashi
        4. PIG-3913-5.patch
          30 kB
          Aniket Namadeo Mokashi
        5. PIG-3913-6.patch
          31 kB
          Aniket Namadeo Mokashi
        6. PIG-3913-7.patch
          30 kB
          Aniket Namadeo Mokashi

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. PIG-3958 TestMRJobStats is broken in 0.13 and trunk

        • Major - Major loss of function.
        • Closed
        is related to

        Bug - A problem which impairs or prevents the functions of the product. PIG-3853 Pig does not kill all the jobs during shutdown for hadoop2

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. PIG-3825 Stats collection needs to be changed for hadoop2 (with auto local mode)

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1214 Add support for counters in Hadoop Local Mode

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            aniket486 Aniket Namadeo Mokashi
            aniket486 Aniket Namadeo Mokashi
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment