Uploaded image for project: 'OODT'
  1. OODT
  2. OODT-698 Adapt OODT Process Streaming Data
  3. OODT-700

Create a Spark Runner for the Wengine

    XMLWordPrintableJSON

    Details

      Description

      Spark is a useful technology for processing of streams and running map-reduce within the memory of a cluster. Thus it is important to fold it into the streaming augmentation of OODT.

      Currently, the wengine runner extension point seems to be the best place to put this. In order for the project to be fully backwards compatible, some processing point in OODT must allow multiplexing of jobs (some are run on mesos via resource manager, some are run in Spark, etc.).

        Attachments

        1. svn-workflow-spark-2014-11-05.txt
          14 kB
          Michael Starch

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              starchmd@umich.edu Michael Starch
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: