Details
Major Description
As a part of STORM-2441 we are separating out the classpaths for the client+worker process from everything else in storm. This is great but it really will make some of our users upset, because it is not a rolling upgrade, and because they will need to recompile their topologies (again).
We have done a really good job in maintaining compatibility with older versions of storm because we use thrift for all communication and state storage. This means that a new supervisor and or nimbus should be able to talk to just about any existing client/worker out there. So we should explicitly support this.
We should add in config options to supervisors.
storm.supported.jvms which is a map of the version of the JVM to the JAVA_HOME path for it.
and
storm.legacy.worker.classpath which is a map of the version of storm to a CLASSPATH that can be used to launch a legacy worker process.
They should be set for all supervisors and nimbus.
Then we also add in some metadata that the client submits to nimbus along with a topology. Namely the version of the storm client they are running on and the version of the JVM they are running on.
Nimbus can then decide (possibly through another config, but probably just through convention with some config overrides) if the version of the client + JVM is compatible with the version of storm + JVM currently on the cluster. If so it should just let it through. If not it should pick a version of the JVM + storm that is compatible. If there are none available it will reject the request. We should also allow end users to set these configs.
For this to work well we need some good version matching/sorting code that is lenient, even during rolling upgrades.
For example if a user submits a topology with a 1.0.3 client to a 2.0.0 cluster. Nimbus sees that it has 1.0.3 installed great it will start running with that, but then we do a rolling upgrade of the cluster and upgrade move to 1.0.4. The supervisors should be able to launch the workers for that topology with a 1.0.4 classpath.
As such part of the worker heartbeat should also include the version of storm + JVM that the worker is running with.
We should display that on the UI and display the version it was submitted with on the UI too.
