[MAPREDUCE-2639] MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: mrv2

Environment

All

• Assignee: Josh Wills
• Reporter: Josh Wills
• Created: 04/Jul/11 00:33
• Updated: 29/Jul/11 22:41

Description

A catch-all JIRA for a pass I took through the v2.app.speculate package.

1) Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined in TestRuntimeEstimators.
2) Fixed two bugs in DataStatistics: 1) a divide by zero in the variance calculation in the case that count == 0 and 2) a synchronization issue in how the updateStatistics method was implemented,
3) A bunch of typo corrections, formatting fixes, and adding some consistency around the null value checking.

I probably need to do a couple more passes through this code to get it into better shape, but this seemed like a good start. Will attach my patch momentarily.

Issue Links

• is related to: MAPREDUCE-279 Map-Reduce 2.0

Sub-Tasks

• Fix the ExponentiallySmoothedTaskRuntimeEstimator and its unit test: Closed
• Fix two bugs in v2.app.speculate.DataStatistics: Resolved
• Fixing typos/formatting/null checking in v2.app.speculate package: Open