Pig
  1. Pig
  2. PIG-3981

Use Tez counter to collect per output stats

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.15.0
    • Component/s: tez
    • Labels:
      None

      Description

      Pig currently goes to hdfs to collect those information. That will take from several hundred milliseconds to several seconds, which adds overhead for namenode and hurt performance especially for iterative queries.

        Issue Links

          Activity

          Daniel Dai made changes -
          Fix Version/s 0.15.0 [ 12328760 ]
          Fix Version/s 0.14.0 [ 12326954 ]
          Daniel Dai made changes -
          Field Original Value New Value
          Link This issue is blocked by TEZ-1173 [ TEZ-1173 ]
          Daniel Dai created issue -

            People

            • Assignee:
              Daniel Dai
              Reporter:
              Daniel Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development