0. Use Java API
1. Create any custom ML transformer
2. Make it MLReadable and MLWritable
3. Add to pipeline
4. Evaluate model, e.g. CrossValidationModel, and save results to disk
5. For custom transformer you can use DefaultParamsReader and DefaultParamsWriter, for instance
6. Load model from saved directory
7. All out-of-the-box objects are loaded successfully, e.g. Pipeline, Evaluator, etc.
8. Your custom transformer will fail with NPE
In Java this only works for static methods.
As we are implementing MLReadable or MLWritable, then this call should be instance method call.
- relates to
SPARK-19498 Discussion: Making MLlib APIs extensible for 3rd party libraries