Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1284

Code generation for operator fusion

    XMLWordPrintableJSON

Details

    • Code generation for operator fusion

    Description

      This epic aims to introduce code generation capabilities for automatic operator fusion, which helps to reduce the number of materialized intermediates, reduce the number of input scans, exploit sparsity, and reduce compute, while significantly reducing the development effort compared to hand-coded fused operators.

      For the 1.0 release, we will introduce code generation as an experimental feature, covering an extended version of SPOOF's code generator as described in the following paper:

      T. Elgamal, S. Luo, M. Boehm, A. V. Evfimievski, S. Tatikonda, B. Reinwald, P. Sen: SPOOF: Sum-Product Optimization and Operator Fusion for Large-Scale Machine Learning, CIDR, 2017
      http://cidrdb.org/cidr2017/papers/p3-elgamal-cidr17.pdf

      Attachments

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-1594 Mlogit performance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is duplicated by

          Task - A task that needs to be done. SYSTEMDS-448 Source code generation for automatic operator fusion

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue SYSTEMDS-1828 New simplification rewrite for merging sequences blocks

          • Major - Major loss of function.
          • Closed

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SYSTEMDS-2065 Code generation for operator fusion v3

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2019 Perftest multiple algorithms failing w/ codegen and intercept 2

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2020 Perftest bivariate statistics w/ codegen causes unnecessary sparse-dense conversions

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2021 Perftest stratstats w/ codegen getting stuck in optimization

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2022 Perftest LinregCG and MLogreg failing w/ codegen due to dim mismatch

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2024 Perftest Mlogreg multinomial, icp 1 failing w/ index-out-of-bounds

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2026 Perftest OOM on genStratStatsData 80GB w/ codegen

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. SYSTEMDS-2028 Perftest Mlogreg 10M x 1K, icpt 1 codegen failing w/ unsupported cbind in cell

          • Major - Major loss of function.
          • Closed

          Activity

            People

              mboehm7 Matthias Boehm
              mboehm7 Matthias Boehm
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: