Pig
  1. Pig
  2. PIG-2032

Penny related multiple improvements/issues.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
      None
    • Environment:

      All env

      Description

      This is from a mail I sent Ben and Chris in feb

      Essentially the changes are :

      a) Allow for progress reporting so that long running agent api invocations dont result in killing the task.

      b) Decouple creation from initialization of monitor agent harness.

      c) I increased the threads for coordinator since ibis has slightly io bound coordinator messages to be processed : but this might not be generally relevant.
      Maybe some way to configure it would be good ? (The default value was observed to be exhausted really fast for some invocations of ibis !)

      d) Handle shutdown more gracefully at the agent side - monitor and wait for relevant future's to be complete.
      Note that the code for this is slightly sensitive in terms of how it is written since there are idioms in netty which it supports (immediate invocation vs deferred invocation on future completion).

      1. diffs
        20 kB
        Mridul Muralidharan

        Activity

        Hide
        Mridul Muralidharan added a comment -

        Not cleaned the file as per pig requirements, etc ... Similarly, no tests attached for the changes.

        Ibis depends on these changes, and it works on top of this - so it has been indirectly tested.

        Show
        Mridul Muralidharan added a comment - Not cleaned the file as per pig requirements, etc ... Similarly, no tests attached for the changes. Ibis depends on these changes, and it works on top of this - so it has been indirectly tested.
        Hide
        Olga Natkovich added a comment -

        Since we already branch for 0.9, I don't believe and penny improvements will make it in.

        In general, for improvements to a particular feature to make it to a particular release it is good to have resources lined to work on the feature.

        Show
        Olga Natkovich added a comment - Since we already branch for 0.9, I don't believe and penny improvements will make it in. In general, for improvements to a particular feature to make it to a particular release it is good to have resources lined to work on the feature.
        Hide
        Dmitriy V. Ryaboy added a comment -

        I thought we aren't linking features to releases now until they go in, since releases are now time-bound, not feature-bound.

        Show
        Dmitriy V. Ryaboy added a comment - I thought we aren't linking features to releases now until they go in, since releases are now time-bound, not feature-bound.
        Hide
        Olga Natkovich added a comment -

        But we need to give some time for stabilizing the code before we release. So once we branch, it should be bug fixes, not enhancements

        Show
        Olga Natkovich added a comment - But we need to give some time for stabilizing the code before we release. So once we branch, it should be bug fixes, not enhancements
        Hide
        Dmitriy V. Ryaboy added a comment -

        Oh right, I just meant about tying it to 0.10...

        Show
        Dmitriy V. Ryaboy added a comment - Oh right, I just meant about tying it to 0.10...
        Hide
        Dmitriy V. Ryaboy added a comment -

        You didn't tie it to 0.10. I'm seeing things. Never mind.

        Show
        Dmitriy V. Ryaboy added a comment - You didn't tie it to 0.10. I'm seeing things. Never mind.
        Hide
        Mridul Muralidharan added a comment -

        Please note that the changes are not to pig, but to penny codebase alone : and without these, it fails for non-trivial usecases. I will leave it to Ben though ...

        Show
        Mridul Muralidharan added a comment - Please note that the changes are not to pig, but to penny codebase alone : and without these, it fails for non-trivial usecases. I will leave it to Ben though ...

          People

          • Assignee:
            Unassigned
            Reporter:
            Mridul Muralidharan
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development