The Motivations and changes made to the api (illustrated above) -
1. Simplify the interface by reducing classes that are exposed in the user api, and some additional steps. - Removed the class ReadEntity , WriteEntity.
2. In original user api, some functions in HcatWriter and HcatReader were to used only in the master, and others in the slaves. - Made the entire class functionality available in slaves by removing the use of functions HcatWriter and HcatReader by user in master.
3. The interface for reading exposed InputSplit as the unit of work. Users might want more information than what is provided by InputSplit, and also different number of InputSplits, and different ways of combining the InputSplit (eg divide inputs on partition values). Replaced InputSplit with a SlaveReaderContext as way to divide the input. SlaveReaderContext can potentially add more functions in future that help in getting metadata information about the unit of work.
4. In future, for reading, user should be able to specify different ways of splitting the data across slaves (eg by partition column value). User should also be able to use custom logic for splitting data across slaves. The use of SlaveReaderContext will help with this.
5. The writer API has similar change to use SlaveWriterContext .