Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3981

Use Tez counter to collect per output stats

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: tez
    • Labels:
      None

      Description

      Pig currently goes to hdfs to collect those information. That will take from several hundred milliseconds to several seconds, which adds overhead for namenode and hurt performance especially for iterative queries.

        Issue Links

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              daijy Daniel Dai
              Reporter:
              daijy Daniel Dai
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development