Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2104 A CrossProductEdge which produces synthetic cross-product parallelism
  3. TEZ-3458

Auto grouping for cartesian product edge(unpartitioned case)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • None
    • None
    • Reviewed

    Description

      Original CartesianProductVertexManagerUnpartitioned set parallelism as product of all source vertices parallelism which may explode to insane number. We should do auto reduce as in ShuffleVertexManager to avoid this.

      Attachments

        1. TEZ-3458.1.patch
          76 kB
          Zhiyuan Yang
        2. TEZ-3458.2.patch
          76 kB
          Zhiyuan Yang
        3. TEZ-3458.3.patch
          76 kB
          Zhiyuan Yang
        4. TEZ-3458.4.patch
          68 kB
          Zhiyuan Yang
        5. TEZ-3458.5.patch
          70 kB
          Zhiyuan Yang
        6. TEZ-3458.6.patch
          71 kB
          Zhiyuan Yang
        7. TEZ-3458.additional.patch
          1.0 kB
          Zhiyuan Yang

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue TEZ-3573 Allow user to cap number of task in cartesian product (unpartitioned case)

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              zhiyuany Zhiyuan Yang
              zhiyuany Zhiyuan Yang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: