River
  1. River
  2. RIVER-316

RFC Library, Application & Class Versioning, Dynamically Mobile Codebases and Classloading enhancements

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: net_jini_loader
    • Labels:
      None
    • Environment:

      All

      Description

      Request for Comments:

      Proposal to add support for Dynamic Mobile Codebases and Application fine grained class versioning as well as Coarse grained Library versioning , to enable River User devolopers, to provide distinction between classes with the same fully qualified class name when code differences created by refactoring packages or library updates break backward compatibility between classes contained within that library or package. ClassWorlds can be used to segregate ClassRealms for application packages and different library versions.

      A dependency tree array object (contains dependency references between classes, fully qualified class names are stored as String objects) returned by the new ClassDepend tool (replacement of classdep functionality) may be suitable (with some modification) for recording class versioning, for later navigation of the codebase for class version verification, perhaps this could be stored in serialized form with the codebase.

      The ASM library might be used to modify existing, externally sourced library class file bytecodes to add a LIBRARYVERSIONID static final field, with an accessor method, for library code used in codebases, to mark the class files with the library release version.

      serialVersionUID (when it exists), along with the CLASSVERSION static field, might be used to determine the dependency and backward compatibility of classes in a codebase, this information could be stored in the dependency tree along with the CLASSVERSION, fully qualified class name and class file checksum.

      1. classworlds-1.0-src.zip
        82 kB
        Peter Firmstone
      2. Java Classloader issues relating to Jini smli_tr-2006-149.pdf
        359 kB
        Peter Firmstone
      3. VersionedDynamicClassesRev6.tgz
        61 kB
        Peter Firmstone
      4. VersionedDynamicClassesRev7.tgz
        64 kB
        Peter Firmstone
      5. VersionedDynamicClassesRev8.tgz
        165 kB
        Peter Firmstone

        Issue Links

          Activity

          Hide
          Peter Firmstone added a comment -

          A very thorough Research paper from 2006, detailing ClassLoader issues with Java RMI and JINI, with suggested solutions, authored by Michael Warres, uploaded with permission, courtesy Michael and Sun Microsystems.

          Show
          Peter Firmstone added a comment - A very thorough Research paper from 2006, detailing ClassLoader issues with Java RMI and JINI, with suggested solutions, authored by Michael Warres, uploaded with permission, courtesy Michael and Sun Microsystems.
          Hide
          Peter Firmstone added a comment -

          How about something totally different?

          A codebase lookup service perhaps? Given a class name, serialVersionUID and class file checksum it can return a list of URL's registered as providing that service and containing that class and return a new checksum if the class file has been updated in a compatible manner, a lookup table could be provided and another service to propagate updated class file checksums relevant to a particular class name and serialVersionUID between codebase lookup services.

          A classfile checksum would need to be generated for each class (when?) and sent with the marshalled object, instead of a list of codebase URL's

          A background process might also check the currently loaded class files periodically for updates, and update it? How could this be done?

          Class file caching proxy's could also register as codebases with the lookup service, in order to reduce remote network traffic and preserve class files when codebases are down?

          I'd be interested in what Dennis Reedy has to say about QOS?

          Show
          Peter Firmstone added a comment - How about something totally different? A codebase lookup service perhaps? Given a class name, serialVersionUID and class file checksum it can return a list of URL's registered as providing that service and containing that class and return a new checksum if the class file has been updated in a compatible manner, a lookup table could be provided and another service to propagate updated class file checksums relevant to a particular class name and serialVersionUID between codebase lookup services. A classfile checksum would need to be generated for each class (when?) and sent with the marshalled object, instead of a list of codebase URL's A background process might also check the currently loaded class files periodically for updates, and update it? How could this be done? Class file caching proxy's could also register as codebases with the lookup service, in order to reduce remote network traffic and preserve class files when codebases are down? I'd be interested in what Dennis Reedy has to say about QOS?
          Hide
          Peter Firmstone added a comment - - edited

          An Object's runtime Class Type is the fully qualified class name + the originating ClassLoader.

          HTTP codebase's are part of the problem, the URLClassLoader is fixed in the object's Type (class identity), which may change over time. Michael Warres addressed this problem by creating a dynamic codebase service where the URL was a cryptographic hash checksum of the jar file (stored data) identity.

          Michael made a presentation on Service based codebases, apparently not much code was required to implement it. We cannot directly copy the code (interfaces etc) from the presentation due to copyright, although we can produce functionally equivalent code.

          http://www.jini.org/files/meetings/eighth/presentations/Warres/Warres.pdf

          So the dynamic service based codebase could move around, and be offered redundantly.

          In addition, we could update/upgrade/replace the Hierarchical based PreferredClassLoader relationship with a more flexible DynamicVersioningClassLoader based on ClassWorlds to segregate incompatible class packages while granting compatible classes the ability to communicate. There is BSD licensed code that we could build on:

          http://classworlds.codehaus.org/ ClassWorlds has some very elegant simple models (easy to code & use) that may help us.

          From the above website: "The ClassWorlds model does away with the hierarchy normally associated with ClassLoaders. Instead, there is a pool of ClassRealms <http://classworlds.codehaus.org/apidocs/com/codehaus/classworlds/ClassRealm.html> which can import arbitrary packages from other ClassRealms. Effectively, ClassWorlds turns the old-style hierarchy into a directed graph."

          One might give a library its own ClassLoader in each JVM for instance, then we could make that library available to the applications / services that depended upon it. A later version of that library would have a separate ClassLoader so that "Jar hell" or "Classpath Hell" (standalone JVM talk) or its distributed equivalent "Class Type Hell" or "ClassLoader Hell" are avoided (unfair: ClassLoaders are a blessing in disguise, they add another dimension to namespaces).

          The new com.sun.jini.tool.classdepend package has functionality to record class dependency relationships in an array, this could be modified to also store each classes unique sha-1 or md5 hash checksum, as well as serialVersionUID's, if they implement Serialisable or Externalizable, this dependency array or a similar class name string indexed HashMap could be returned on request as part of the codebase service.

          I purchased a copy of the following paper online (Thank's Jim Waldo for the tip, no endorsements implied), I found a freely available copy you can all read. Its called Modular Software Upgrades for Distributed Sytems by Sameer Ajmant, Barbara Liskov and Liuba Shrira. It discusses updating services in distributed systems. It is a course grained versioning system.

          http://www.pmg.csail.mit.edu/~ajmani/papers/ecoop06-upgrades.pdf

          So I'm currently trying to get my head around a new ClassLoader framework for classes where a developer knows which objects need to be preserved over time, while their underlying class implementations may change, this would be a fine grained versioning system, at the Class Level. The new package com.sun.jini.tools.classdepend can return a dependency tree stored in an array, this could be extended to record each classes unique sha-1 or md5 hash checksum, as well as serialVersionUID's, if they implement Serialisable or Externalizable and if a Class implements VersionedClass, it already stores the fully qualified class name. The sha-1 or md5 hash checksum would also form part of Security of downloaded code as this could be checked before loading the Class file.

          Versioning, Identity and preservation of object state / contents over time is much harder and is unsolved. Jim Waldo recommends Classes implement interfaces, where interfaces are used for Type. This allows objects from different ClassLoaders that implement the same interface to interact and be interchanged. For instance if you have a Class called Money.class (version 1) and you reimplement its methods and add additional methods in Money.class(version 2), the only way the objects can be used in the same array etc is if they share a common Interface or ancestor class. If you inherit Money.class (version 1) and call it MoneyTree.class you can override all replaced methods and add additional methods and it can be used as a money object, however you can't use the new additional methods when in company with the original class Money's objects, only those that existed prior.

          Working towards solving this problem, I'm thinking about a ClassLoader Versioning Framework for Objects we want to distribute and preserve over time, preserving their state and contents while retaining the ability to upgrade their class files or bytecodes (different class versions with the same fully qualified class name will have different Type's and cannot coexist in the same ClassLoader). Interfaces alone, enable polymorphic interoperability between objects with different types.

          Serialization can be used to migrate object state between class file bytecodes (marshall, unmarshall into a replacement ClassLoader) with required visibility granted by what we can build on using ClassWorlds.

          Identity is more difficult, for objects who's identity is sufficiently determined by the equals() and hashCode() methods this should be sufficient. However objects that require a unique identity can be broken down again into two types:

          1. Immutable objects where Object.equals() doesn't determine object Identity.
          2. Mutable objects where Object.equals() doesn't determine object Identity.

          I think #1 could be handled by an ObjectUniqueID service that provides three unique numbers (or two separate services? a TimeService and a RandomNumberGeneratorService); The time in milliseconds at the time of request and two random numbers, one could be reirieved locally. The object would receive this service at instantiation time if required. The likelihood that two random numbers and the time in milliseconds would produce a match for the same fully quallified class name would be vary rare indeed. It may be desireable for all VersionedObjects to contain unique ID fields, versionObjectCreationTimeID, versionRemoteRandomID and versionLocalRandomID.

          Well #2 would be more complex, distributed objects would need a Transaction Manager, as well as implement the requirements of #1.

          when checking for VersionedObject.equals(), this must check prior to any other calcs that the object is an instance of each interface implemented.

          VersionedObject's, could be instantiated by:

          public interface VersioningClassLoader {

          public MobileObjectDelegate instantiate(Builder builderObject, String fullyQualifiedClassName){};

          }

          The builder Object's Class would have to implement the same interfaces as the VersionedClass.

          public Constructors are not permitted as these return strong references, in order to prevent the JVM providing one, they must implement a private constructor only.

          Each VersionedClassLoader would be garbage collected after all VersionedObjects it manages. Objects and Class files could be grouped by their Java Package. Each Package could have it's own ClassLoader.

          A package upgrade check could be triggered upon receiving a distributed object with a new class file version, or receiving an ObsoleteMethodException.

          All objects referenced by VersionedClass Objects and declared transient, would be considered supporting objects and would go out of scope once the VersionedClass Objects do, for example a new class file upgrade might dictate another version of an external library, new objects required by the VersionedObject would have to be created during unmarshalling and the new class files for the library downloaded via the codebase Service, the codebase service would also advertise via the service, the dependency tree it contains, so the VersioningClassLoader Framework would determine the Libraries suitability and make it available via a Library version specific ClassLoader Realm with ClassWorlds if it isn't already available.

          All remote objects upon unmarshalling would be checked to see if their class currently exists in a ClassLoader and if the Object already resides in memory based on Lease Validity, identity or equality and if so the in memory object used.

          A Versioned Object would only be permitted to keep references to objects that meet any of the following criteria:

          1. Non versioned and Non Serializable objects that are declared transient and can be reconstructed after unmarshalling. OR
          2. JVM native implementations that implement Serializable or Externalizable. OR
          3. Package Private Versioned Objects, A MobileObjectDelegate, or other VersionedObjects. OR
          4. Objects from non versioned Classes that are not subject to change and implement serializable.

          Once the basic VersionedClass, MobileObjectDelegate, DynamicVersionedClassLoader interfaces and implementations agreed upon, a number of useful utility VersionedClass implementations could be created, for developers to utilise, preferably Interfaces and their implementations for immutable objects, such as Currency, Quantity, Unit, Money, etc.

          What I'm talking about is conceptual and experimental, I'm hoping others will be able to provide some thoughts / input, assist and see if we can't produce something useful along with the changes you've made and lessons learned. Or alternatively tell me I'm totally nuts

          Please raise any issues you can see so they can be addressed, the more earlier on the better.

          Show
          Peter Firmstone added a comment - - edited An Object's runtime Class Type is the fully qualified class name + the originating ClassLoader. HTTP codebase's are part of the problem, the URLClassLoader is fixed in the object's Type (class identity), which may change over time. Michael Warres addressed this problem by creating a dynamic codebase service where the URL was a cryptographic hash checksum of the jar file (stored data) identity. Michael made a presentation on Service based codebases, apparently not much code was required to implement it. We cannot directly copy the code (interfaces etc) from the presentation due to copyright, although we can produce functionally equivalent code. http://www.jini.org/files/meetings/eighth/presentations/Warres/Warres.pdf So the dynamic service based codebase could move around, and be offered redundantly. In addition, we could update/upgrade/replace the Hierarchical based PreferredClassLoader relationship with a more flexible DynamicVersioningClassLoader based on ClassWorlds to segregate incompatible class packages while granting compatible classes the ability to communicate. There is BSD licensed code that we could build on: http://classworlds.codehaus.org/ ClassWorlds has some very elegant simple models (easy to code & use) that may help us. From the above website: "The ClassWorlds model does away with the hierarchy normally associated with ClassLoaders. Instead, there is a pool of ClassRealms < http://classworlds.codehaus.org/apidocs/com/codehaus/classworlds/ClassRealm.html > which can import arbitrary packages from other ClassRealms. Effectively, ClassWorlds turns the old-style hierarchy into a directed graph." One might give a library its own ClassLoader in each JVM for instance, then we could make that library available to the applications / services that depended upon it. A later version of that library would have a separate ClassLoader so that "Jar hell" or "Classpath Hell" (standalone JVM talk) or its distributed equivalent "Class Type Hell" or "ClassLoader Hell" are avoided (unfair: ClassLoaders are a blessing in disguise, they add another dimension to namespaces). The new com.sun.jini.tool.classdepend package has functionality to record class dependency relationships in an array, this could be modified to also store each classes unique sha-1 or md5 hash checksum, as well as serialVersionUID's, if they implement Serialisable or Externalizable, this dependency array or a similar class name string indexed HashMap could be returned on request as part of the codebase service. I purchased a copy of the following paper online (Thank's Jim Waldo for the tip, no endorsements implied), I found a freely available copy you can all read. Its called Modular Software Upgrades for Distributed Sytems by Sameer Ajmant, Barbara Liskov and Liuba Shrira. It discusses updating services in distributed systems. It is a course grained versioning system. http://www.pmg.csail.mit.edu/~ajmani/papers/ecoop06-upgrades.pdf So I'm currently trying to get my head around a new ClassLoader framework for classes where a developer knows which objects need to be preserved over time, while their underlying class implementations may change, this would be a fine grained versioning system, at the Class Level. The new package com.sun.jini.tools.classdepend can return a dependency tree stored in an array, this could be extended to record each classes unique sha-1 or md5 hash checksum, as well as serialVersionUID's, if they implement Serialisable or Externalizable and if a Class implements VersionedClass, it already stores the fully qualified class name. The sha-1 or md5 hash checksum would also form part of Security of downloaded code as this could be checked before loading the Class file. Versioning, Identity and preservation of object state / contents over time is much harder and is unsolved. Jim Waldo recommends Classes implement interfaces, where interfaces are used for Type. This allows objects from different ClassLoaders that implement the same interface to interact and be interchanged. For instance if you have a Class called Money.class (version 1) and you reimplement its methods and add additional methods in Money.class(version 2), the only way the objects can be used in the same array etc is if they share a common Interface or ancestor class. If you inherit Money.class (version 1) and call it MoneyTree.class you can override all replaced methods and add additional methods and it can be used as a money object, however you can't use the new additional methods when in company with the original class Money's objects, only those that existed prior. Working towards solving this problem, I'm thinking about a ClassLoader Versioning Framework for Objects we want to distribute and preserve over time, preserving their state and contents while retaining the ability to upgrade their class files or bytecodes (different class versions with the same fully qualified class name will have different Type's and cannot coexist in the same ClassLoader). Interfaces alone, enable polymorphic interoperability between objects with different types. Serialization can be used to migrate object state between class file bytecodes (marshall, unmarshall into a replacement ClassLoader) with required visibility granted by what we can build on using ClassWorlds. Identity is more difficult, for objects who's identity is sufficiently determined by the equals() and hashCode() methods this should be sufficient. However objects that require a unique identity can be broken down again into two types: 1. Immutable objects where Object.equals() doesn't determine object Identity. 2. Mutable objects where Object.equals() doesn't determine object Identity. I think #1 could be handled by an ObjectUniqueID service that provides three unique numbers (or two separate services? a TimeService and a RandomNumberGeneratorService); The time in milliseconds at the time of request and two random numbers, one could be reirieved locally. The object would receive this service at instantiation time if required. The likelihood that two random numbers and the time in milliseconds would produce a match for the same fully quallified class name would be vary rare indeed. It may be desireable for all VersionedObjects to contain unique ID fields, versionObjectCreationTimeID, versionRemoteRandomID and versionLocalRandomID. Well #2 would be more complex, distributed objects would need a Transaction Manager, as well as implement the requirements of #1. when checking for VersionedObject.equals(), this must check prior to any other calcs that the object is an instance of each interface implemented. VersionedObject's, could be instantiated by: public interface VersioningClassLoader { public MobileObjectDelegate instantiate(Builder builderObject, String fullyQualifiedClassName){}; } The builder Object's Class would have to implement the same interfaces as the VersionedClass. public Constructors are not permitted as these return strong references, in order to prevent the JVM providing one, they must implement a private constructor only. Each VersionedClassLoader would be garbage collected after all VersionedObjects it manages. Objects and Class files could be grouped by their Java Package. Each Package could have it's own ClassLoader. A package upgrade check could be triggered upon receiving a distributed object with a new class file version, or receiving an ObsoleteMethodException. All objects referenced by VersionedClass Objects and declared transient, would be considered supporting objects and would go out of scope once the VersionedClass Objects do, for example a new class file upgrade might dictate another version of an external library, new objects required by the VersionedObject would have to be created during unmarshalling and the new class files for the library downloaded via the codebase Service, the codebase service would also advertise via the service, the dependency tree it contains, so the VersioningClassLoader Framework would determine the Libraries suitability and make it available via a Library version specific ClassLoader Realm with ClassWorlds if it isn't already available. All remote objects upon unmarshalling would be checked to see if their class currently exists in a ClassLoader and if the Object already resides in memory based on Lease Validity, identity or equality and if so the in memory object used. A Versioned Object would only be permitted to keep references to objects that meet any of the following criteria: 1. Non versioned and Non Serializable objects that are declared transient and can be reconstructed after unmarshalling. OR 2. JVM native implementations that implement Serializable or Externalizable. OR 3. Package Private Versioned Objects, A MobileObjectDelegate, or other VersionedObjects. OR 4. Objects from non versioned Classes that are not subject to change and implement serializable. Once the basic VersionedClass, MobileObjectDelegate, DynamicVersionedClassLoader interfaces and implementations agreed upon, a number of useful utility VersionedClass implementations could be created, for developers to utilise, preferably Interfaces and their implementations for immutable objects, such as Currency, Quantity, Unit, Money, etc. What I'm talking about is conceptual and experimental, I'm hoping others will be able to provide some thoughts / input, assist and see if we can't produce something useful along with the changes you've made and lessons learned. Or alternatively tell me I'm totally nuts Please raise any issues you can see so they can be addressed, the more earlier on the better.
          Hide
          Peter Firmstone added a comment -

          Versioned Dynamic Class Interfaces.

          Please review and comment on Interfaces for Dynamic versioning of Classes.

          I've changed the Terminology from VersionedObject to VersionedClass, it is the class file that changes, the intent is to preserve objects during class file updates.

          ClassWorlds (also attached) is BSD licensed, I'm not sure if it should be integrated or not, review only at this stage. ClassWorlds provides a poor man's subprocess isolation, seeing as the release of the subprocess isolation API is not forthcoming.

          Show
          Peter Firmstone added a comment - Versioned Dynamic Class Interfaces. Please review and comment on Interfaces for Dynamic versioning of Classes. I've changed the Terminology from VersionedObject to VersionedClass, it is the class file that changes, the intent is to preserve objects during class file updates. ClassWorlds (also attached) is BSD licensed, I'm not sure if it should be integrated or not, review only at this stage. ClassWorlds provides a poor man's subprocess isolation, seeing as the release of the subprocess isolation API is not forthcoming.
          Hide
          Peter Firmstone added a comment -

          More Interfaces, more comments

          Show
          Peter Firmstone added a comment - More Interfaces, more comments
          Hide
          Peter Firmstone added a comment -

          From the river-dev list, hopefully Michal, gets approval for licensing, I'm very interested in the services acting in a P2P fashion.

          Michal Kleczek wrote:
          >
          > We have implemented a similar solution in XPro.
          > The main difference is that in our implementation there is no "put" operation
          > in a ContentAddressedStore interface. In other words:
          > 1. Each service publishes a separate ContentAddressStore and becomes its own
          > codebase server.
          > 2. You can run specialized services that cache resources from other
          > ContentAddressStores to achieve HA
          > 3. Optionally you can configure the services to act in a P2P fashion - the
          > resources downloaded by a service are also published.
          >
          > The main motivation behind this was to be able to run services with minimal
          > dependencies on other services being available on the network.
          >
          > After resolving some licencing problems (which should not be very complicated
          > I think) we could contribute this code to River.
          >
          > Thanks,
          > Michal
          >

          Show
          Peter Firmstone added a comment - From the river-dev list, hopefully Michal, gets approval for licensing, I'm very interested in the services acting in a P2P fashion. Michal Kleczek wrote: > > We have implemented a similar solution in XPro. > The main difference is that in our implementation there is no "put" operation > in a ContentAddressedStore interface. In other words: > 1. Each service publishes a separate ContentAddressStore and becomes its own > codebase server. > 2. You can run specialized services that cache resources from other > ContentAddressStores to achieve HA > 3. Optionally you can configure the services to act in a P2P fashion - the > resources downloaded by a service are also published. > > The main motivation behind this was to be able to run services with minimal > dependencies on other services being available on the network. > > After resolving some licencing problems (which should not be very complicated > I think) we could contribute this code to River. > > Thanks, > Michal >
          Hide
          Peter Firmstone added a comment -

          I'm experimenting with some interfaces for programmers to be able to upgrade their application's classfiles dynamically in a djinn, the API will depend on a DynamicVersioningClassLoader that migrates all objects in a package from one ClassRealm (ClassWorlds Terminology) to another, when at least one remote object that is eligible for an updated class file is received, or a package Lease expires and upgraded class files are discovered.

          The interfaces that fundamentally model the types of Objects and Class files that may exist in a Package are:

          VersionedPublicClass extends ObjectIdentity, ObjectPreservedOnUpdate, VersionedClass - For classes that are publicly visible within a djinn.
          VersionedPackagePrivateClass extends ObjectPreservedOnUpdate, VersionedClass - For package private classes that need their object state preserved on upgrades, but don't require visibility outside their Package's ClassRealm.
          VersionedClass - A version identity for the programmer, also used for Classes that don't require their state preserved, to determine later upgrade versions within a package.

          ObjectPreservedOnUpdate - extends Serializable - Indicates that the Object's contents are to be preserved during transition between ClassLoaders
          ObjectIdentity - A unique identifier for a distributable object, a metaphor to local JVM objects that have their instance and reference for identity.

          MobileObjectDelegate - A Wraper implementation of the interface methods of the corresponding VersionedPublicClass implementation object it wraps, its responsibility is to track the VersionedPublicClass Object's movement between ClassRealms (classloaders) and request class file update's from the DynamicVersionClassLoader. The DynamicVersionClassLoader will be responsible for keeping a WeakHashMap of MobileObjectDelegates to notify other MobileObjectDelegates to request their class file updates within a package. I haven't decided on the rules for creation of MobileObjectDelegates to distinguish between new unmarshalled objects and the update of class files for Objects that already exist locally.

          VersionedPublicClass - Designed to be extended only by other interfaces that a programmer develops, it cannot be implemented directly as an Object's Type will change when it transitions between Package ClassRealm's (ClassLoader's). This restriction is limited solely to VersionedPublicClass.

          Builder<Interface> - An Builder interface for VersionedPublicClass implementations, so Class / Object implementations don't form dependency's on implementations of interfaces extending VersionedPublicClass, that would occur from strong references returned by constructors, depending upon the interface instead. One Builder for a particular interface might be suitable for use in many implementing Classes. Note that a Builder implementation can implement multiple interfaces also.

          Show
          Peter Firmstone added a comment - I'm experimenting with some interfaces for programmers to be able to upgrade their application's classfiles dynamically in a djinn, the API will depend on a DynamicVersioningClassLoader that migrates all objects in a package from one ClassRealm (ClassWorlds Terminology) to another, when at least one remote object that is eligible for an updated class file is received, or a package Lease expires and upgraded class files are discovered. The interfaces that fundamentally model the types of Objects and Class files that may exist in a Package are: VersionedPublicClass extends ObjectIdentity, ObjectPreservedOnUpdate, VersionedClass - For classes that are publicly visible within a djinn. VersionedPackagePrivateClass extends ObjectPreservedOnUpdate, VersionedClass - For package private classes that need their object state preserved on upgrades, but don't require visibility outside their Package's ClassRealm. VersionedClass - A version identity for the programmer, also used for Classes that don't require their state preserved, to determine later upgrade versions within a package. ObjectPreservedOnUpdate - extends Serializable - Indicates that the Object's contents are to be preserved during transition between ClassLoaders ObjectIdentity - A unique identifier for a distributable object, a metaphor to local JVM objects that have their instance and reference for identity. MobileObjectDelegate - A Wraper implementation of the interface methods of the corresponding VersionedPublicClass implementation object it wraps, its responsibility is to track the VersionedPublicClass Object's movement between ClassRealms (classloaders) and request class file update's from the DynamicVersionClassLoader. The DynamicVersionClassLoader will be responsible for keeping a WeakHashMap of MobileObjectDelegates to notify other MobileObjectDelegates to request their class file updates within a package. I haven't decided on the rules for creation of MobileObjectDelegates to distinguish between new unmarshalled objects and the update of class files for Objects that already exist locally. VersionedPublicClass - Designed to be extended only by other interfaces that a programmer develops, it cannot be implemented directly as an Object's Type will change when it transitions between Package ClassRealm's (ClassLoader's). This restriction is limited solely to VersionedPublicClass. Builder<Interface> - An Builder interface for VersionedPublicClass implementations, so Class / Object implementations don't form dependency's on implementations of interfaces extending VersionedPublicClass, that would occur from strong references returned by constructors, depending upon the interface instead. One Builder for a particular interface might be suitable for use in many implementing Classes. Note that a Builder implementation can implement multiple interfaces also.
          Hide
          Peter Firmstone added a comment -

          More interface modifications, have a look see what you think.

          Note Fine grained Versioning (Class and Package level) is intended for Application code, not Libraries. I believe that Course grained versioning mechanism should be developed for Libraries.

          Show
          Peter Firmstone added a comment - More interface modifications, have a look see what you think. Note Fine grained Versioning (Class and Package level) is intended for Application code, not Libraries. I believe that Course grained versioning mechanism should be developed for Libraries.
          Hide
          Peter Firmstone added a comment -

          Please have a look tell me what you think, what changes can I make to simplify, what issues am I overlooking. Is there something else I could be taking advantage of?

          Show
          Peter Firmstone added a comment - Please have a look tell me what you think, what changes can I make to simplify, what issues am I overlooking. Is there something else I could be taking advantage of?
          Hide
          Peter Firmstone added a comment -

          Correctly Licensed

          Show
          Peter Firmstone added a comment - Correctly Licensed
          Hide
          Peter Firmstone added a comment -

          Please go through the javadoc, check that I haven't missed anything. I'll check the model myself later, to make sure my latest thoughts are correctly recorded.

          Show
          Peter Firmstone added a comment - Please go through the javadoc, check that I haven't missed anything. I'll check the model myself later, to make sure my latest thoughts are correctly recorded.
          Hide
          Peter Firmstone added a comment - - edited

          Very rough, needs more eyes and further revision, note the MobileObjectDelegate proxy interface has been renamed to PersistentObjectDelegate. Persistent Object's are objects that persist over package upgrades in memory, Serialization is used to upgrade the PersistentObject to the new class implementation. Serialization is not expected to support unmarshalling to old class versions, instead obsolete interface methods can throw ObsoleteMethodException, that can be translated into a RemoteMethodException for remote classes, local classes are expected to check for later class file updates and trigger a Package upgrade request.

          The package upgrades themselves might be managed by an upgrade service that allows the upgrade process to be scheduled for nodes in a djinn.

          Show
          Peter Firmstone added a comment - - edited Very rough, needs more eyes and further revision, note the MobileObjectDelegate proxy interface has been renamed to PersistentObjectDelegate. Persistent Object's are objects that persist over package upgrades in memory, Serialization is used to upgrade the PersistentObject to the new class implementation. Serialization is not expected to support unmarshalling to old class versions, instead obsolete interface methods can throw ObsoleteMethodException, that can be translated into a RemoteMethodException for remote classes, local classes are expected to check for later class file updates and trigger a Package upgrade request. The package upgrades themselves might be managed by an upgrade service that allows the upgrade process to be scheduled for nodes in a djinn.
          Hide
          Peter Firmstone added a comment -

          Name changes interface cleanups etc.

          Show
          Peter Firmstone added a comment - Name changes interface cleanups etc.
          Hide
          Peter Firmstone added a comment -

          I'm thinking about how I can utilize the existing code within River for the Package upgrade process.

          I've renamed the PersistentObject interface to EvolvingObject to more accurately reflect that object state and methods can change, although I've retained the name PersistentObjectDelegate for the proxy interface that represents it. All links to a Java Package's Public API interface implementations are routed through PersistentObjectDelegate's

          The Package Upgrade process has some parallels with Remote Proxy's, so there's opportunity for code reuse.

          The basic theory is:

          All Objects outside a package, remote or local make all calls and reference the packages proxy's that remain in the main application Classloader. The implementation Objects for an entire package are moved between Classloaders that are made visible to the main application Classloader, when some or all of the classes in the package are updated, the class file versions are distinguished by implementing the VersionedClass interface. The proxy is updated with the latest Object reference after a successful upgrade.

          The MarshalledInstance state snapshot of the Package could be stored on disk in the event of failure during the upgrade process also.

          net.jini.io.MarshalledInstance can be utilised for the Serialized state of each object during upgrade, the dynamic codebase service can be passed in via MarshalOuputStream.writeAnnotation(String codebaseService);

          The code base can be related back to the package name and a package version number, this package will of course rely on other package versions, Perahaps a package version static class can contain the Package version number along with the dependency tree? The dependency tree contains all the fully qualified class names for all provider classes, these can be used to determine which packages are required.

          I've changed the name of the entire versioning package from org.apache.river.version to org.apache.river.evolution.

          Should I require a jar archive for each application package? Libraries could have multiple packages and exist in one jar and classloader, I'm still thinking about how to distinguish the two lookup methods and how to get library information into the dependency analysis tool? Perhaps using a configuration file?

          Knowing the required class files in advance means the dynamic code base service could provide only those class files necessary (compressed of course).

          The local Classloader would need to check the existence locally of all class files in the dependency array. This seems too much responsibility for a Classloader however, so I'm open to suggestions on how to split the class loader up.

          Cheers,

          Peter.

          Show
          Peter Firmstone added a comment - I'm thinking about how I can utilize the existing code within River for the Package upgrade process. I've renamed the PersistentObject interface to EvolvingObject to more accurately reflect that object state and methods can change, although I've retained the name PersistentObjectDelegate for the proxy interface that represents it. All links to a Java Package's Public API interface implementations are routed through PersistentObjectDelegate's The Package Upgrade process has some parallels with Remote Proxy's, so there's opportunity for code reuse. The basic theory is: All Objects outside a package, remote or local make all calls and reference the packages proxy's that remain in the main application Classloader. The implementation Objects for an entire package are moved between Classloaders that are made visible to the main application Classloader, when some or all of the classes in the package are updated, the class file versions are distinguished by implementing the VersionedClass interface. The proxy is updated with the latest Object reference after a successful upgrade. The MarshalledInstance state snapshot of the Package could be stored on disk in the event of failure during the upgrade process also. net.jini.io.MarshalledInstance can be utilised for the Serialized state of each object during upgrade, the dynamic codebase service can be passed in via MarshalOuputStream.writeAnnotation(String codebaseService); The code base can be related back to the package name and a package version number, this package will of course rely on other package versions, Perahaps a package version static class can contain the Package version number along with the dependency tree? The dependency tree contains all the fully qualified class names for all provider classes, these can be used to determine which packages are required. I've changed the name of the entire versioning package from org.apache.river.version to org.apache.river.evolution. Should I require a jar archive for each application package? Libraries could have multiple packages and exist in one jar and classloader, I'm still thinking about how to distinguish the two lookup methods and how to get library information into the dependency analysis tool? Perhaps using a configuration file? Knowing the required class files in advance means the dynamic code base service could provide only those class files necessary (compressed of course). The local Classloader would need to check the existence locally of all class files in the dependency array. This seems too much responsibility for a Classloader however, so I'm open to suggestions on how to split the class loader up. Cheers, Peter.
          Hide
          Peter Firmstone added a comment -

          Some interface refactoring, more needed of course. I've pilfered the Dynamic Proxy from Harmony's java.lang.reflect package along with the test cases, the tests are passing, ready for modification. Harmony's Proxy will be modified to support the new Delegate for Evolving Objects, such that the Delegate can be generated dynamically as well as control the synchronization policies (ClassUpgradePolicy) during the Package upgrade process.

          I'm now wondering how to inject the Exporter interface into the Proxy as well, for Remote Proxy's & Stub's

          From there I'll look at concurrency support in Classworlds and how to implement the codebase service and dependency analysis.

          Cheers,

          Peter.

          Show
          Peter Firmstone added a comment - Some interface refactoring, more needed of course. I've pilfered the Dynamic Proxy from Harmony's java.lang.reflect package along with the test cases, the tests are passing, ready for modification. Harmony's Proxy will be modified to support the new Delegate for Evolving Objects, such that the Delegate can be generated dynamically as well as control the synchronization policies (ClassUpgradePolicy) during the Package upgrade process. I'm now wondering how to inject the Exporter interface into the Proxy as well, for Remote Proxy's & Stub's From there I'll look at concurrency support in Classworlds and how to implement the codebase service and dependency analysis. Cheers, Peter.

            People

            • Assignee:
              Unassigned
              Reporter:
              Peter Firmstone
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:

                Development