Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2020

For 1-1 edge vertex configured event may be sent incorrectly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For a fully configured 1-1 vertex configured event will be sent after init because the vertex manager does not signal intent to reconfigure. However a change in parallelism upstream on the 1-1 edge can cause this vertex's parallelism to be updated accordingly. This breaks the guarantee that configured should be sent after vertex is fully configured. This is because update of parallelism on 1-1 edge has until now been hard-coded in the Vertex code. This can be removed now with the InputReadyVertexManager handling the 1-1 edge and the addition of vertex state updates. Not only will this fix the issue but also remove the special handling for 1-1 edges in the AM which should not be there for cleanliness.

        Attachments

        1. TEZ-2020.1.patch
          33 kB
          Bikas Saha
        2. synthetic_job.png
          82 kB
          Rajesh Balamohan

          Issue Links

            Activity

              People

              • Assignee:
                bikassaha Bikas Saha
                Reporter:
                bikassaha Bikas Saha
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: