Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.22.0
-
Component/s: contrib/gridmix
-
Labels:None
-
Hadoop Flags:Reviewed
Description
We observed the following exception occasionally at the end of the Gridmix run:
Exception in thread "StatsCollectorThread" java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at org.apache.hadoop.mapred.gridmix.Statistics$StatCollector.updateAndNotifyClusterStatsListeners(Statistics.java:220)
at org.apache.hadoop.mapred.gridmix.Statistics$StatCollector.run(Statistics.java:205)
Activity
| Field | Original Value | New Value |
|---|---|---|
| Assignee | Hong Tang [ hong.tang ] |
Trivial patch that uses CopyOnWriteArrayList to avoid concurrent modification. No unit test included as it is hard to reproduce the synchronization bug through unit tests.
| Attachment | mr-1961-20100723.patch [ 12450368 ] |
| Status | Open [ 1 ] | Patch Available [ 10002 ] |
| Status | Patch Available [ 10002 ] | Open [ 1 ] |
| Status | Open [ 1 ] | Patch Available [ 10002 ] |
| Component/s | contrib/gridmix [ 12313086 ] |
+1 for the patch.
Show
Ranjit Mathew
added a comment - +1 for the patch.
test-patch output:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no new tests are needed for this patch.
[exec] Also please list what manual steps were performed to verify this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 tests included
The patch fixes a concurrency bug, and is hard to write unit test to cover. The patch is tested with real benchmark runs and has shown to be effective.
Show
Hong Tang
added a comment - test-patch output:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no new tests are needed for this patch.
[exec] Also please list what manual steps were performed to verify this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 tests included
The patch fixes a concurrency bug, and is hard to write unit test to cover. The patch is tested with real benchmark runs and has shown to be effective.
+1
I committed this. Thanks, Hong!
Show
Chris Douglas
added a comment - +1
I committed this. Thanks, Hong!
| Status | Patch Available [ 10002 ] | Resolved [ 5 ] |
| Hadoop Flags | [Reviewed] | |
| Fix Version/s | 0.22.0 [ 12314184 ] | |
| Resolution | Fixed [ 1 ] |
Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)
| Status | Resolved [ 5 ] | Closed [ 6 ] |
It seems that StatsCollectorThread is still active when we call "clusterStatlisteners.clear();" from "Statistics.shutdown()".
A simple fix is to replace "clusterStatlisteners.clear();" with "lusterStatlisteners = new ArrayList<StatListener<ClusterStats>>()".
While we are at it, we probably should also fix the same problem with jobStatListeners.