Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-16

Implement collection of data from executor to client

    XMLWordPrintableJSON

    Details

      Description

      Our current implementation simply serializes and writes files from the executor, and the client reads the data to collect the RDD data. This should be handled more properly.
      Please refer to edu.snu.nemo.compiler.frontend.spark.core.java.SparkFrontendUtils and edu.snu.nemo.compiler.frontend.spark.transform.CollectTransform for more information.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wonook Won Wook Song
                Reporter:
                wonook Won Wook Song
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: