Perform usability testing of SystemML, ideally with assistance from IBM Design.
Audit the usability of the system from the perspective of several classes of users:
- Library user: Treats SystemML as a collection of canned algorithms
- Light customization user: Uses one of the provided algorithms as a starting point, but makes minor modifications to cover a specific use case
- Heavy customization user: Writes a DML script from scratch
For all types of users, the testing should cover the entire data science life cycle:
- Small-scale testing on a laptop
- Large-scale testing on a cluster
- Deployment in a production system
After gathering information from this testing, document issues and develop a plan for resolving these issues.