Apache S4
  1. Apache S4
  2. S4-67

S4R packaging: improve packaging of dependencies

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      Currently, the creation of the S4R package involves unpackaging all dependencies, then putting them altogether in a single file along with the S4 application classes.

      This is very inefficient (and therefore potentially long, resulting in long development cycles) because there is a lot of file copying involved. In addition it may be brittle due to name conflicts.

      We should improve the process by:

      • including external jars in the S4R archive without unpackaging them, typically in the /lib dir
      • updating the classloading mechanism in S4. Ideas could be taken from Hadoop's RunJar class. We should avoid fancy stuff here, and the current multi-classloader is not really needed if we have a single app per S4 node.
      • "shading" platform classes and dependencies, in order to avoid any conflict between S4 platform dependencies and application dependencies?

        Activity

        Hide
        Matthieu Morel added a comment -

        S4R packaging and classloading improvements were implemented in S4-72.
        Library shading is a separate ticket S4-80

        Show
        Matthieu Morel added a comment - S4R packaging and classloading improvements were implemented in S4-72 . Library shading is a separate ticket S4-80

          People

          • Assignee:
            Matthieu Morel
            Reporter:
            Matthieu Morel
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development