Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
0.7.0
-
None
-
None
-
None
Description
In a running container, Samza framework classes and user job classes are bundled into the same classpath. This can cause conflicts, especially with transitive dependencies. For example, samza-yarn depends on hadoop-common which depends on Guava, and the user job may depend on some database client library which depends on a different version of Guava. The ensuing version conflicts can be nasty.
The same problem also plagues other frameworks that run user code, for example MapReduce and Storm. To my knowledge, neither of those have solved the problem. Tomcat has solved the problem by using a separate classloader for every webapp running in its container.
Storm's ideas for classpath isolation are discussed here and here. It's tracked in STORM-129.
It would be worth investigating this — if classpath isolation requires an intrusive change to Samza, it might be worth getting the pain out of the way while Samza is still young.