Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2612 Critical path analyzer for DAGs
  3. TEZ-2647

Add input causality dependency for attempts

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.8.0-alpha
    • None
    • None
    • Reviewed

    Description

      Attempts can have input dependencies on the producer task attempts that produced the data being consumed by the attempt.
      DataMovement events capture this dependency. In the interest of space, we need to be able to capture the dependency that matters - the one that provided the last data for the input to complete.
      For starters, we could
      1) have the system track the last data movement event that was sent to an attempt
      2) then have the inputs be able to report the last relevant data movement event

      Attachments

        1. TEZ-2647.3.patch
          52 kB
          Bikas Saha
        2. TEZ-2647.2.patch
          280 kB
          Bikas Saha
        3. TEZ-2647.1.patch
          51 kB
          Bikas Saha

        Activity

          People

            bikassaha Bikas Saha
            bikassaha Bikas Saha
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: