Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1821

Improve the training process in mnist_lenet_distrib_sgd.dml

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • SystemML 1.0.0
    • None
    • ParFor, Runtime
    • None

    Description

      For Remote_Spark mode, SystemML engine will export the intermediate matrixes into HDFS, which may impair the performance of distributed model training. Therefore the unnecessary intermediate matrixes need to be avoided, and instead we can directly use the dimension index to compute the final batch matrix.

      Attachments

        Issue Links

          is a child of

          Task - A task that needs to be done. SYSTEMDS-1809 Optimize the performance of the distributed MNIST_LeNet_Sgd model training

          • Major - Major loss of function.
          • Open

          Activity

            People

              Tenma Fei Hu
              Tenma Fei Hu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: