[S4-67] S4R packaging: improve packaging of dependencies

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.5.0
• Assignee: Matthieu Morel
• Reporter: Matthieu Morel
• Created: 27/Jun/12 10:39
• Updated: 20/Jul/12 07:36
• Resolved: 20/Jul/12 07:36

Description

Currently, the creation of the S4R package involves unpackaging all dependencies, then putting them altogether in a single file along with the S4 application classes.

This is very inefficient (and therefore potentially long, resulting in long development cycles) because there is a lot of file copying involved. In addition it may be brittle due to name conflicts.

We should improve the process by:

• including external jars in the S4R archive without unpackaging them, typically in the /lib dir
• updating the classloading mechanism in S4. Ideas could be taken from Hadoop's RunJar class. We should avoid fancy stuff here, and the current multi-classloader is not really needed if we have a single app per S4 node.
• "shading" platform classes and dependencies, in order to avoid any conflict between S4 platform dependencies and application dependencies?