Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1612

ShuffleVertexManager's EdgeManager should not hard code source num tasks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.5.0
    • 0.5.1
    • None
    • None
    • Reviewed

    Description

      Several Pig unit tests hang intermittently. For example, TestNewPlanImplicitSplit.testImplicitSplitInCoGroup, which is a DAG of 4 nodes:

      It uses auto-parallelism, vertex 106 change parallelism from 2->1, and vertex 107 from 21->1.

      Log attached.

      Attachments

        1. syslog_dag_1411413615885_0001_1
          73 kB
          Daniel Dai
        2. DAG1.png
          15 kB
          Daniel Dai
        3. testfail1.log.tar.gz
          16 kB
          Daniel Dai
        4. runwithmaster.tar.gz
          2.52 MB
          Daniel Dai
        5. TEZ-1612.1.patch
          5 kB
          Bikas Saha

        Activity

          People

            bikassaha Bikas Saha
            daijy Daniel Dai
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: