Pig
  1. Pig
  2. PIG-1436

Print number of records outputted at each step of a Pig script

    Details

    • Tags:
      pig hadoop records output debug grunt mc hammer funky pants

      Description

      I often run a script multiple times, or have to go and look through Hadoop task logs, to figure out where I broke a long script in such a way that I get 0 records out of it. I think this is a common problem.

      If someone can point me in the right direction, I can make a pass at this.

        Activity

        Hide
        Alan Gates added a comment -

        Russell,

        Richard's already doing a lot of work in this area. Check out PIG-1389, PIG-908, PIG-864, PIG-809 to see if those will meet your needs. If not, please discuss with him as his current project is to add script usage statistics.

        Show
        Alan Gates added a comment - Russell, Richard's already doing a lot of work in this area. Check out PIG-1389 , PIG-908 , PIG-864 , PIG-809 to see if those will meet your needs. If not, please discuss with him as his current project is to add script usage statistics.
        Hide
        Olga Natkovich added a comment -

        I think Richard is already doing this as part of his stats work

        Show
        Olga Natkovich added a comment - I think Richard is already doing this as part of his stats work
        Hide
        Richard Ding added a comment -

        Russell,

        PIG-1478 implemented a callback mechanism that allows users to retrieve stats after each job. Will this meet your needs?

        Show
        Richard Ding added a comment - Russell, PIG-1478 implemented a callback mechanism that allows users to retrieve stats after each job. Will this meet your needs?
        Hide
        Olga Natkovich added a comment -

        This looks like duplicate of PIG-1478. Please, re-open if this is not the case

        Show
        Olga Natkovich added a comment - This looks like duplicate of PIG-1478 . Please, re-open if this is not the case

          People

          • Assignee:
            Richard Ding
            Reporter:
            Russell Jurney
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development