Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      The abacus examples were not moved to aggregate when it was created. Provide an aggregate example which shows users how to use the aggregate library.

      As a part of this issue move TestAggregates to the org.apache.hadoop.mapred.lib.aggregate package.

      1. patch_1547.txt
        27 kB
        Runping Qi

        Issue Links

          Activity

          Tom White created issue -
          Tom White made changes -
          Field Original Value New Value
          Link This issue blocks HADOOP-1302 [ HADOOP-1302 ]
          Runping Qi made changes -
          Link This issue depends upon HADOOP-1521 [ HADOOP-1521 ]
          Hide
          Runping Qi added a comment -

          A proper example will not work before HADOOP-1521 is fixed.

          Show
          Runping Qi added a comment - A proper example will not work before HADOOP-1521 is fixed.
          Hide
          Runping Qi added a comment -

          This patch addresses the following issues:

          1. Move the TestAggregate to mapred.lib.aggregate package.

          2. Provide an example in the examples dir.
          The example is to use Aggregate to count the word occurrences in the input files.

          3. Replace work Abacus in the comments in the source files with Aggregate

          4. Modify the ValueAggregatorJob so that it allows the user to specify a user jar file
          in the spec file under attribute anme "user.jar.file".
          This makes Aggregate package work without depending on HADOOP-1521

          Show
          Runping Qi added a comment - This patch addresses the following issues: 1. Move the TestAggregate to mapred.lib.aggregate package. 2. Provide an example in the examples dir. The example is to use Aggregate to count the word occurrences in the input files. 3. Replace work Abacus in the comments in the source files with Aggregate 4. Modify the ValueAggregatorJob so that it allows the user to specify a user jar file in the spec file under attribute anme "user.jar.file". This makes Aggregate package work without depending on HADOOP-1521
          Runping Qi made changes -
          Attachment patch_1547.txt [ 12361439 ]
          Runping Qi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Show
          Hadoop QA added a comment - +1 http://issues.apache.org/jira/secure/attachment/12361439/patch_1547.txt applied and successfully tested against trunk revision r554144. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/testReport/ Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/379/console
          Hide
          Runping Qi added a comment -

          I hope this patch can get in the next release.
          Let me know if there are any concerns for committing it.

          Show
          Runping Qi added a comment - I hope this patch can get in the next release. Let me know if there are any concerns for committing it.
          Hide
          Owen O'Malley added a comment -

          Runping, can you please reformat the patch to stay within 80 characters?

          Show
          Owen O'Malley added a comment - Runping, can you please reformat the patch to stay within 80 characters?
          Owen O'Malley made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Tom White added a comment -

          Runping, please also update the usage line in the javadoc of AggregateWordCount to include

          1 textinputformat
          

          to specify the number of reducers and the type of input. When I tried omitting these I got an exception.

          Show
          Tom White added a comment - Runping, please also update the usage line in the javadoc of AggregateWordCount to include 1 textinputformat to specify the number of reducers and the type of input. When I tried omitting these I got an exception.
          Runping Qi made changes -
          Attachment patch_1547.txt [ 12361439 ]
          Hide
          Runping Qi added a comment -


          A new patch addressing Tom and Owen's comments attached.

          Show
          Runping Qi added a comment - A new patch addressing Tom and Owen's comments attached.
          Runping Qi made changes -
          Attachment patch_1547.txt [ 12361701 ]
          Runping Qi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Tom White added a comment -

          I've just committed this. Thanks Runping!

          I made a further minor change to the usage javadoc.

          Show
          Tom White added a comment - I've just committed this. Thanks Runping! I made a further minor change to the usage javadoc.
          Tom White made changes -
          Fix Version/s 0.14.0 [ 12312474 ]
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hide
          Hadoop QA added a comment -

          -1, could not apply patch.

          The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12361701/patch_1547.txt as a patch to trunk revision r555770.

          Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/401/console

          Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

          Show
          Hadoop QA added a comment - -1, could not apply patch. The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12361701/patch_1547.txt as a patch to trunk revision r555770. Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/401/console Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-Nightly #154 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/154/ )
          Doug Cutting made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s examples [ 12311574 ]
          Jeff Hammerbacher made changes -
          Link This issue is related to HADOOP-908 [ HADOOP-908 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          2d 19h 15m 1 Owen O'Malley 12/Jul/07 17:16
          Open Open Patch Available Patch Available
          10d 9h 54m 2 Runping Qi 12/Jul/07 19:00
          Patch Available Patch Available Resolved Resolved
          2h 37m 1 Tom White 12/Jul/07 21:38
          Resolved Resolved Closed Closed
          38d 21h 34m 1 Doug Cutting 20/Aug/07 19:12

            People

            • Assignee:
              Runping Qi
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development