Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-60

IR-based dynamic optimization

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.2
    • None

    Description

      We should adapt DataSkewPolicy to use compiler optimizer as a single point of optimization to align with our philosophy of DAG structure-aware optimization.

      Currently, DataSkewRuntimePass optimizes PhysicalPlan instead of IR DAG in dynamic repartitioning of shuffle data. We need to fix this to optimize via IR DAG instead.

      Attachments

        Issue Links

          Activity

            People

              jeongyoon Jeongyoon Eo
              jeongyoon Jeongyoon Eo
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: