Details
-
Improvement
-
Status: Done
-
Major
-
Resolution: Done
-
None
-
None
-
None
-
None
Description
Both of loader and s2counter_loader are depended on custom spark sub-directory, which contains helpers for spark job and spark streaming job with spark 1.4.1 dependencies. It seems that there are no actual needs to keep spark sub-project under s2graph and other sub-project to depends on this.
Suggest to version up the spark from 1.4.1 to 2.3.0(latest) and refactor `loader` and `s2counter_loader` to depends on latest version.
Also, I don't see any reason to separate s2counter_loader and loader so suggest to merge these two into one sub-project.
Attachments
Issue Links
- links to
