[TAVERNA-941] Use Docker microservices from workflow

• Type: Story
• Status: To Do
• Priority: Major
• Resolution: Unresolved
• Component/s: Taverna Engine
• Assignee: Unassigned
• Reporter: Stian Soiland-Reyes
• Created: 18/Mar/16 11:07
• Updated: 18/Mar/16 11:07

Description

While TAVERNA-901 suggests a way to run Docker command line tools from Taverna, a more interesting aspect could be to run microservices from Taverna - with bindings to the dynamic ports of Docker.

This would allow usage of existing WSDL and REST services in a Taverna workflow, but with an alternative Profile to use local Docker microservices.
This could then use server-based web UI tools like OpenRefine and Jupyter - but also pure servers like RServe, ElasticSearch and Jena Fuseki SPARQL endpoints.

Taken to the extreme this could be to start up a local Open PHACTS stack (multiple Docker images combined with docker-compose). (As this requires 32 GB of RAM and 0.5 TB of disk space that would not be a microservice though!)

This is more architecturally challenging than the activity of TAVERNA-901 - as it would mean preprocessing steps like

• Workflow Bundle lists required docker images
• Fetch Docker images
  • Optional: load images stored in workflow bundle
• Start Docker containers - possibly listening on dynamic ports
• Optional: Using docker-compose for combining many containers
• Check that containers are "ready" (unsolved problem in Docker world)
• Modify/generate workflow profile to use the assigned localhost/port bindings for the docker microservices
• Run the rest of the workflow
• Shut down containers
• Remove containers (optional)
• Store provenance about Docker containers into Docker Bundle (optional)
  • Snapshot docker images used into Data Bundle

Issue Links

• relates to: TAVERNA-901 Run Docker from Taverna