Pig
  1. Pig
  2. PIG-2664

Allow PPNL impls to get more job info during the run

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None

      Description

      Implementations of PigProcessNotificationListener don't have access to much job-related into until after the job is completed. A few minor edits to PigStatsUtil can change that.

      • Expose the JobClient
      • Set an initial JobStats object upon kicking off a job. This makes JobId available to clients once it's assigned.
      1. PIG-2664.2.patch
        4 kB
        Bill Graham
      2. PIG-2664.1.patch
        3 kB
        Bill Graham

        Activity

        Bill Graham made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Bill Graham made changes -
        Attachment PIG-2851.1.patch [ 12538449 ]
        Bill Graham made changes -
        Attachment PIG-2851.1.patch [ 12538449 ]
        Bill Graham made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.11 [ 12318878 ]
        Resolution Fixed [ 1 ]
        Hide
        Bill Graham added a comment -

        Committed.

        Show
        Bill Graham added a comment - Committed.
        Hide
        Julien Le Dem added a comment -

        +1

        Show
        Julien Le Dem added a comment - +1
        Bill Graham made changes -
        Attachment PIG-2664.2.patch [ 12525241 ]
        Hide
        Bill Graham added a comment -

        Attaching patch 2, which has getJobClient() as an abstract method in ScriptStats and implemented by SimpleJobStats.

        Show
        Bill Graham added a comment - Attaching patch 2, which has getJobClient() as an abstract method in ScriptStats and implemented by SimpleJobStats.
        Hide
        Bill Graham added a comment -

        There's also EmbeddedPigStats which is used when Pig is running in embedded mode. It also subclasses PigStats but throws UnsupportedOperationException exceptions for most of the abstract methods it implements. We can follow the same approach with an abstract getJobClient() method.

        I also don't love the ((SimplePigStats)PigStats.get()).getFoo() approach, but it's the pattern already in use in PigStatsUtil and other options didn't seem great. I'm ok with moving getJobClient it to PigStats though. That way we can at least get rid of the cast.

        Show
        Bill Graham added a comment - There's also EmbeddedPigStats which is used when Pig is running in embedded mode. It also subclasses PigStats but throws UnsupportedOperationException exceptions for most of the abstract methods it implements. We can follow the same approach with an abstract getJobClient() method. I also don't love the ((SimplePigStats)PigStats.get()).getFoo() approach, but it's the pattern already in use in PigStatsUtil and other options didn't seem great. I'm ok with moving getJobClient it to PigStats though. That way we can at least get rid of the cast.
        Hide
        Julien Le Dem added a comment -

        In what case is PigStats.get() not a SimplePigStats ? should we just add getJobClient() to PigStats ?

        Show
        Julien Le Dem added a comment - In what case is PigStats.get() not a SimplePigStats ? should we just add getJobClient() to PigStats ?
        Bill Graham made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Bill Graham made changes -
        Field Original Value New Value
        Attachment PIG-2664.1.patch [ 12523937 ]
        Hide
        Bill Graham added a comment -

        Here's a patch for review.

        Show
        Bill Graham added a comment - Here's a patch for review.
        Bill Graham created issue -

          People

          • Assignee:
            Bill Graham
            Reporter:
            Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development