Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Invalid
-
SystemDS 3.1
-
None
-
None
Description
Generally we recommend a memory configuration of -Xmx<X>g -Xms<Y>g -Xmn<Z>g, where Y=X and Z=0.1*X. However, our sparkDML.sh only sets up the driver and executor memory which only affects X. We should use spark.driver.extraJavaOptions and spark.executor.extraJavaOptions to configure the remaining options. reinwald@us.ibm.com could you take a look into this?
Attachments
Issue Links
- Is contained by
-
SYSTEMDS-1217 Perftest 0.12 release and related improvements
- Closed
- is related to
-
SYSTEMDS-455 OOM CP transpose in Spark hybrid mode
- Closed