Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
In distributed spark backend, pass a given pre-trained model to the paramserv function may cause the data inconsistency. Because the pre-trained model would be cached in driver's memory. In this case, when kicking off the paramserv func, the workers firstly will try to read the data from HDFS where the dirty data in pre-trained model has not been persisted. This leads to a inconsistency. So the idea is to export the dirty data to HDFS before kicking off the remote workers.