Pig
  1. Pig
  2. PIG-2964

Add helper method getJobList() to PigStats.JobGraph. Extend visibility of couple methods on same class

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None
    • Tags:
      API

      Description

      PigStats.JobGraph currently provides an iterator. The rationale behind providing a List getter is that it simply provides a lot more methods than an iterator. One could do that in the caller code, but providing this helper in the API would be beneficial. For eg, a framework that is used by clients could initiate several pig scripts/store commands at once. At the framework layer, you might want to be able to determine the number of MR jobs in total spawned by these multiple scripts and query stats on those. That's just one use-case, there could be more methods on List that a user could be interested in.

      1. PIG-2964.patch
        4 kB
        Prashant Kommireddi
      2. PIG-2964_1.patch
        6 kB
        Prashant Kommireddi

        Activity

        Prashant Kommireddi created issue -
        Hide
        Prashant Kommireddi added a comment -

        Added test case to TestPigStats. Also cleaned up code in the Test class, re-using FileUtils from commons.io and eliminating the explicit code.

        Show
        Prashant Kommireddi added a comment - Added test case to TestPigStats. Also cleaned up code in the Test class, re-using FileUtils from commons.io and eliminating the explicit code.
        Prashant Kommireddi made changes -
        Field Original Value New Value
        Attachment PIG-2964.patch [ 12548717 ]
        Prashant Kommireddi made changes -
        Description The rationale behind providing a List getter is that it simply provides a lot more methods than an iterator. One could do that in the caller code, providing this helper in the API would be beneficial. For eg, a framework that is used by clients could initiate several pig scripts/store commands at once. At the framework layer, you might want to be able to determine the number of MR jobs in total spawned by these multiple scripts and query stats on those. That's just one use-case, there could be more methods on List that a user could be interested in. PigStats.JobGraph currently provides an iterator. The rationale behind providing a List getter is that it simply provides a lot more methods than an iterator. One could do that in the caller code, but providing this helper in the API would be beneficial. For eg, a framework that is used by clients could initiate several pig scripts/store commands at once. At the framework layer, you might want to be able to determine the number of MR jobs in total spawned by these multiple scripts and query stats on those. That's just one use-case, there could be more methods on List that a user could be interested in.
        Hide
        Bill Graham added a comment -

        Looks good, but would you please make two minor edits. The temp file should include the name of the class + the method instead of the JIRA. That way people know who generated it if they come across it. Also, instead of System.out.println we should use commons logging.

        Show
        Bill Graham added a comment - Looks good, but would you please make two minor edits. The temp file should include the name of the class + the method instead of the JIRA. That way people know who generated it if they come across it. Also, instead of System.out.println we should use commons logging.
        Bill Graham made changes -
        Patch Info Patch Available [ 10042 ]
        Hide
        Prashant Kommireddi added a comment -

        I added commons logging and removed some not so good code "printStackTrace()" from the existing tests on TestPigStats.

        Show
        Prashant Kommireddi added a comment - I added commons logging and removed some not so good code "printStackTrace()" from the existing tests on TestPigStats.
        Prashant Kommireddi made changes -
        Attachment PIG-2964_1.patch [ 12548761 ]
        Hide
        Bill Graham added a comment -

        Committed, thanks Prashant!

        Show
        Bill Graham added a comment - Committed, thanks Prashant!
        Bill Graham made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Prashant Kommireddi added a comment -

        Thanks for the review, Bill.

        Show
        Prashant Kommireddi added a comment - Thanks for the review, Bill.
        Bill Graham made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        16h 57m 1 Bill Graham 12/Oct/12 01:48
        Resolved Resolved Closed Closed
        133d 4h 4m 1 Bill Graham 22/Feb/13 04:53

          People

          • Assignee:
            Prashant Kommireddi
            Reporter:
            Prashant Kommireddi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development