Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      The abacus examples were not moved to aggregate when it was created. Provide an aggregate example which shows users how to use the aggregate library.

      As a part of this issue move TestAggregates to the org.apache.hadoop.mapred.lib.aggregate package.

      1. patch_1547.txt
        27 kB
        Runping Qi

        Issue Links

          Activity

          Hide
          Runping Qi added a comment -

          A proper example will not work before HADOOP-1521 is fixed.

          Show
          Runping Qi added a comment - A proper example will not work before HADOOP-1521 is fixed.
          Hide
          Runping Qi added a comment -

          This patch addresses the following issues:

          1. Move the TestAggregate to mapred.lib.aggregate package.

          2. Provide an example in the examples dir.
          The example is to use Aggregate to count the word occurrences in the input files.

          3. Replace work Abacus in the comments in the source files with Aggregate

          4. Modify the ValueAggregatorJob so that it allows the user to specify a user jar file
          in the spec file under attribute anme "user.jar.file".
          This makes Aggregate package work without depending on HADOOP-1521

          Show
          Runping Qi added a comment - This patch addresses the following issues: 1. Move the TestAggregate to mapred.lib.aggregate package. 2. Provide an example in the examples dir. The example is to use Aggregate to count the word occurrences in the input files. 3. Replace work Abacus in the comments in the source files with Aggregate 4. Modify the ValueAggregatorJob so that it allows the user to specify a user jar file in the spec file under attribute anme "user.jar.file". This makes Aggregate package work without depending on HADOOP-1521
          Show
          Hadoop QA added a comment - +1 http://issues.apache.org/jira/secure/attachment/12361439/patch_1547.txt applied and successfully tested against trunk revision r554144. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/testReport/ Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/console
          Hide
          Runping Qi added a comment -

          I hope this patch can get in the next release.
          Let me know if there are any concerns for committing it.

          Show
          Runping Qi added a comment - I hope this patch can get in the next release. Let me know if there are any concerns for committing it.
          Hide
          Owen O'Malley added a comment -

          Runping, can you please reformat the patch to stay within 80 characters?

          Show
          Owen O'Malley added a comment - Runping, can you please reformat the patch to stay within 80 characters?
          Hide
          Tom White added a comment -

          Runping, please also update the usage line in the javadoc of AggregateWordCount to include

          1 textinputformat
          

          to specify the number of reducers and the type of input. When I tried omitting these I got an exception.

          Show
          Tom White added a comment - Runping, please also update the usage line in the javadoc of AggregateWordCount to include 1 textinputformat to specify the number of reducers and the type of input. When I tried omitting these I got an exception.
          Hide
          Runping Qi added a comment -


          A new patch addressing Tom and Owen's comments attached.

          Show
          Runping Qi added a comment - A new patch addressing Tom and Owen's comments attached.
          Hide
          Tom White added a comment -

          I've just committed this. Thanks Runping!

          I made a further minor change to the usage javadoc.

          Show
          Tom White added a comment - I've just committed this. Thanks Runping! I made a further minor change to the usage javadoc.
          Hide
          Hadoop QA added a comment -

          -1, could not apply patch.

          The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12361701/patch_1547.txt as a patch to trunk revision r555770.

          Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/401/console

          Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

          Show
          Hadoop QA added a comment - -1, could not apply patch. The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12361701/patch_1547.txt as a patch to trunk revision r555770. Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/401/console Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-Nightly #154 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/154/ )

            People

            • Assignee:
              Runping Qi
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development