Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Currently, out-of-core infrastructure reports some metrics as part of master/worker log, and also as part of Hadoop job counters. There is a need to show, for a specific out-of-core policy, how much of the graph were kept in memory at the worst (i.e. memory heaviest) moment in the execution. Particularly, a user would be interested in knowing how many more machines the job should be run on, so that the entire execution fits in memory.