• Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.0
    • Component/s: None
    • Labels:


      In storm, Kryo (2.21) is used for serialization:

      The user must use the same version storm does, or there will be a java class error at runtime.

      Storm depends on a quasi-abandoned library: carbonite:

      which depends on Kryo 2.21 and Twitter chill 0.3.6:

      Chill, currently on 0.7.3, would like to upgrade to Kryo 3.0.3:

      because Spark, also depending on chill, would like to upgrade for performance improvements and bugfixes.

      Unfortunately, summingbird depends on storm:

      so, if chill is upgraded, and that gets on the classpath, summingbird will break at runtime.

      I propose:
      1) copy the carbonite code into storm. It is likely the only consumer.
      2) bump the storm kryo dependency after chill upgrades: recall that storm actually depends on chill-java. A dependency that could possibly be removed after you pull carbonite in.
      3) once a new version of storm is published, summingbird (and scalding) can upgrade to the latest chill.

      Also, I hope for:
      4) we as a JVM community get better about classpath isolation and versioning. Diamonds like this in one big classpath make large codebases very fragile.


          Issue Links



              • Assignee:
                abhishek.agarwal Abhishek Agarwal
                posco Oscar Boykin
              • Votes:
                0 Vote for this issue
                8 Start watching this issue


                • Created: