Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: tez
    • Labels:
      None

      Description

      Currently, Pig prints DAGStatus for Tez jobs every second. This generates a lot of info message in stderr. In particular, for long running jobs, every second status report seems unnecessary.

      1) Every 10 seconds seems reasonable for most jobs.
      2) But if the default isn't good, users should be able to configure via a property.

      1. PIG-4006-1.patch
        3 kB
        Cheolsoo Park

        Activity

        Hide
        Cheolsoo Park added a comment -

        Committed to trunk. Thank you Daniel for the review!

        Show
        Cheolsoo Park added a comment - Committed to trunk. Thank you Daniel for the review!
        Hide
        Daniel Dai added a comment -

        +1

        Show
        Daniel Dai added a comment - +1
        Hide
        Cheolsoo Park added a comment -

        The attached patch introduces a new property pig.tez.dag.status.report.interval which can be used to configure the interval of DAGStatus info message in seconds. The default value is 10 seconds.

        Show
        Cheolsoo Park added a comment - The attached patch introduces a new property pig.tez.dag.status.report.interval which can be used to configure the interval of DAGStatus info message in seconds. The default value is 10 seconds.

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development