Details
-
Type:
Improvement
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.14.0
-
Component/s: None
-
Labels:None
Description
The abacus examples were not moved to aggregate when it was created. Provide an aggregate example which shows users how to use the aggregate library.
As a part of this issue move TestAggregates to the org.apache.hadoop.mapred.lib.aggregate package.
Issue Links
- blocks
-
HADOOP-1302
Remove deprecated contrib/abacus code
-
- Closed
-
- depends upon
-
MAPREDUCE-1
Hadoop mapreduce should always ship the jar file(s) specified by the user
-
- Resolved
-
- is related to
-
HADOOP-908
Hadoop Abacus, a package for performing simple counting/aggregation
-
- Closed
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
This patch addresses the following issues:
1. Move the TestAggregate to mapred.lib.aggregate package.
2. Provide an example in the examples dir.
The example is to use Aggregate to count the word occurrences in the input files.
3. Replace work Abacus in the comments in the source files with Aggregate
4. Modify the ValueAggregatorJob so that it allows the user to specify a user jar file
in the spec file under attribute anme "user.jar.file".
This makes Aggregate package work without depending on HADOOP-1521
+1
http://issues.apache.org/jira/secure/attachment/12361439/patch_1547.txt applied and successfully tested against trunk revision r554144.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/console
I hope this patch can get in the next release.
Let me know if there are any concerns for committing it.
Runping, can you please reformat the patch to stay within 80 characters?
Runping, please also update the usage line in the javadoc of AggregateWordCount to include
1 textinputformat
to specify the number of reducers and the type of input. When I tried omitting these I got an exception.
A new patch addressing Tom and Owen's comments attached.
I've just committed this. Thanks Runping!
I made a further minor change to the usage javadoc.
-1, could not apply patch.
The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12361701/patch_1547.txt as a patch to trunk revision r555770.
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/401/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
A proper example will not work before
HADOOP-1521is fixed.