Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2664

Allow PPNL impls to get more job info during the run

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None

      Description

      Implementations of PigProcessNotificationListener don't have access to much job-related into until after the job is completed. A few minor edits to PigStatsUtil can change that.

      • Expose the JobClient
      • Set an initial JobStats object upon kicking off a job. This makes JobId available to clients once it's assigned.
      1. PIG-2664.2.patch
        4 kB
        Bill Graham
      2. PIG-2664.1.patch
        3 kB
        Bill Graham

        Activity

        Hide
        billgraham Bill Graham added a comment -

        Here's a patch for review.

        Show
        billgraham Bill Graham added a comment - Here's a patch for review.
        Hide
        julienledem Julien Le Dem added a comment -

        In what case is PigStats.get() not a SimplePigStats ? should we just add getJobClient() to PigStats ?

        Show
        julienledem Julien Le Dem added a comment - In what case is PigStats.get() not a SimplePigStats ? should we just add getJobClient() to PigStats ?
        Hide
        billgraham Bill Graham added a comment -

        There's also EmbeddedPigStats which is used when Pig is running in embedded mode. It also subclasses PigStats but throws UnsupportedOperationException exceptions for most of the abstract methods it implements. We can follow the same approach with an abstract getJobClient() method.

        I also don't love the ((SimplePigStats)PigStats.get()).getFoo() approach, but it's the pattern already in use in PigStatsUtil and other options didn't seem great. I'm ok with moving getJobClient it to PigStats though. That way we can at least get rid of the cast.

        Show
        billgraham Bill Graham added a comment - There's also EmbeddedPigStats which is used when Pig is running in embedded mode. It also subclasses PigStats but throws UnsupportedOperationException exceptions for most of the abstract methods it implements. We can follow the same approach with an abstract getJobClient() method. I also don't love the ((SimplePigStats)PigStats.get()).getFoo() approach, but it's the pattern already in use in PigStatsUtil and other options didn't seem great. I'm ok with moving getJobClient it to PigStats though. That way we can at least get rid of the cast.
        Hide
        billgraham Bill Graham added a comment -

        Attaching patch 2, which has getJobClient() as an abstract method in ScriptStats and implemented by SimpleJobStats.

        Show
        billgraham Bill Graham added a comment - Attaching patch 2, which has getJobClient() as an abstract method in ScriptStats and implemented by SimpleJobStats.
        Hide
        julienledem Julien Le Dem added a comment -

        +1

        Show
        julienledem Julien Le Dem added a comment - +1
        Hide
        billgraham Bill Graham added a comment -

        Committed.

        Show
        billgraham Bill Graham added a comment - Committed.

          People

          • Assignee:
            billgraham Bill Graham
            Reporter:
            billgraham Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development