Details
Description
I think we should move all of the Counters that the framework updates into a single class called TaskMetrics. TaskMetrics would have specific fields for each of the metrics like input records, input bytes, output records, etc.
It would both reduce the serialized size of the heartbeats (by shrinking the Counters down to just the user's counters) and decrease the latency for updates to the JobTracker (since Counters are sent at most 1/minute instead of 1/heartbeat).
Attachments
Attachments
Issue Links
- blocks
-
MAPREDUCE-2102 Collect iops per task
- Open
-
MAPREDUCE-220 Collecting cpu and memory usage for MapReduce tasks
- Closed
-
MAPREDUCE-718 Support for per-phase speculative execution
- Open
- is blocked by
-
MAPREDUCE-917 Remove getInputCounter and getOutputCounter from Contexts
- Closed
- is related to
-
MAPREDUCE-1304 Add counters for task time spent in GC
- Closed
-
MAPREDUCE-2835 Make per-job counter limits configurable
- Closed
- is required by
-
MAPREDUCE-2037 Capturing interim progress times, CPU usage, and memory usage, when tasks reach certain progress thresholds
- Closed
- relates to
-
MAPREDUCE-1500 Improve the way Counters are managed by JobTracker
- Open
-
MAPREDUCE-1173 Documenting MapReduce metrics
- Open
-
MAPREDUCE-2025 Improve job counter wire protocol with incremental schema and delta values
- Open