Hadoop Common
  1. Hadoop Common
  2. HADOOP-3770

improve composition, submission and result collection of gridmix

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: benchmarks
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Current gridmix submits jobs using a set of scripts, which is inconvenient and the results are difficult to collect. To improve the gridmix submission and results collection, we implemented a new program using JobControl to submit and collect the results of jobs
      Also the new gridmix allows to have more different types of jobs such as, pig jobs, jobs with combiners etc.

      1. patch-3770.txt
        175 kB
        Lingyun Yang
      2. patch-3770.txt
        178 kB
        Lingyun Yang
      3. patch-3770.v2.txt
        104 kB
        Runping Qi

        Issue Links

          Activity

          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12386124/patch-3770.txt
          against trunk revision 677264.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 release audit. The applied patch generated 213 release audit warnings (more than the trunk's current 209 warnings).

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/testReport/
          Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/current/releaseAuditDiffWarnings.txt
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12386124/patch-3770.txt against trunk revision 677264. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 213 release audit warnings (more than the trunk's current 209 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2877/console This message is automatically generated.
          Hide
          Owen O'Malley added a comment -

          Please make sure that the new files contain Apache license headers. That is why this patch got a -1 for Release Audit.

          Show
          Owen O'Malley added a comment - Please make sure that the new files contain Apache license headers. That is why this patch got a -1 for Release Audit.
          Hide
          Lingyun Yang added a comment -

          Owen:
          Thanks for the comments. A new patch is attached according to your comments.

          Lingyun

          Show
          Lingyun Yang added a comment - Owen: Thanks for the comments. A new patch is attached according to your comments. Lingyun
          Hide
          Runping Qi added a comment -

          The new gridmix differs from the original gridmix in the following ways:

          1. Use an xml config file to specify the types and sizes mix of a mix load. This provides better granularity control.
          2. Use JobControl to submit gridmix load, instead of shell script.
          3. Include Pig jobs.

          A new patch is attached

          Show
          Runping Qi added a comment - The new gridmix differs from the original gridmix in the following ways: 1. Use an xml config file to specify the types and sizes mix of a mix load. This provides better granularity control. 2. Use JobControl to submit gridmix load, instead of shell script. 3. Include Pig jobs. A new patch is attached
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12393667/patch-3770.v2.txt
          against trunk revision 712923.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The applied patch generated 220 javac compiler warnings (more than the trunk's current 1011 warnings).

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12393667/patch-3770.v2.txt against trunk revision 712923. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 220 javac compiler warnings (more than the trunk's current 1011 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3572/console This message is automatically generated.
          Hide
          Runping Qi added a comment -

          An updated patch.
          Fixed some typo in README.gridmix2
          Factor out the code for creating a streamong job object as a static method in streaming.StreamJob.java

          Show
          Runping Qi added a comment - An updated patch. Fixed some typo in README.gridmix2 Factor out the code for creating a streamong job object as a static method in streaming.StreamJob.java
          Hide
          Runping Qi added a comment -


          Re-submit an updated version

          Show
          Runping Qi added a comment - Re-submit an updated version
          Hide
          Runping Qi added a comment -


          Fix a small bug in the previous version.

          Show
          Runping Qi added a comment - Fix a small bug in the previous version.
          Hide
          Runping Qi added a comment -

          re-submit a modified patch

          Show
          Runping Qi added a comment - re-submit a modified patch
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12393750/patch-3770.v2.txt
          against trunk revision 713423.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12393750/patch-3770.v2.txt against trunk revision 713423. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3580/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          I haven't been over the details, but had a few general suggestions after a first pass:

          • Most of this doesn't conform to the coding guidelines. Converting tabs to spaces, removing commented-out code, putting constants and defaults in a reasonable place, etc. should be done before this can be committed.
          • Temporary directories should be configurable and default off of a single, configurable temp dir rather than being hard-coded off /tmp
          • If independent from the original, the configuration and drivers of gridmix2 should not be in the same place. If this is intended as a replacement for gridmix, it should modify the existing benchmark rather than creating file, file2 pairs. If it's a new benchmark, it should be in src/benchmarks/gridmix2.
          • Would it be possible to split the pig benchmarks into a separate JIRA? This is simply too large to review well.
          • GridMixRunner is unnecessarily enormous. Most of the methods are setting defaults and performing work best encapsulated in the *Creator classes that currently do trivial work. This class would also benefit from utility methods converting results from Configuration::getStrings to int[] (instead of subclassing Configuration), abstracting out the creation of unique Strings for runs (the use of Calendar/Date may not be the correct choice), javadoc, and general cleanup
          • Exceptions are almost always ignored; most probably should not be.
          Show
          Chris Douglas added a comment - I haven't been over the details, but had a few general suggestions after a first pass: Most of this doesn't conform to the coding guidelines. Converting tabs to spaces, removing commented-out code, putting constants and defaults in a reasonable place, etc. should be done before this can be committed. Temporary directories should be configurable and default off of a single, configurable temp dir rather than being hard-coded off /tmp If independent from the original, the configuration and drivers of gridmix2 should not be in the same place. If this is intended as a replacement for gridmix, it should modify the existing benchmark rather than creating file , file2 pairs. If it's a new benchmark, it should be in src/benchmarks/gridmix2. Would it be possible to split the pig benchmarks into a separate JIRA? This is simply too large to review well. GridMixRunner is unnecessarily enormous. Most of the methods are setting defaults and performing work best encapsulated in the *Creator classes that currently do trivial work. This class would also benefit from utility methods converting results from Configuration::getStrings to int[] (instead of subclassing Configuration), abstracting out the creation of unique Strings for runs (the use of Calendar/Date may not be the correct choice), javadoc, and general cleanup Exceptions are almost always ignored; most probably should not be.
          Hide
          Runping Qi added a comment -

          Follow Chris' suggestion to break the issue into two part.
          This Jira will be for the gridmix2 part only. Pig gridmix part will be addressed in a separate Jira.

          A lot of common codes were factored out.
          The formatting problems were fixed.

          Show
          Runping Qi added a comment - Follow Chris' suggestion to break the issue into two part. This Jira will be for the gridmix2 part only. Pig gridmix part will be addressed in a separate Jira. A lot of common codes were factored out. The formatting problems were fixed.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394371/patch-3770.v2.txt
          against trunk revision 719787.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394371/patch-3770.v2.txt against trunk revision 719787. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3631/console This message is automatically generated.
          Hide
          Runping Qi added a comment -

          fix a minor bug

          Show
          Runping Qi added a comment - fix a minor bug
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394496/patch-3770.v2.txt
          against trunk revision 719787.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394496/patch-3770.v2.txt against trunk revision 719787. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3641/console This message is automatically generated.
          Hide
          Runping Qi added a comment -

          Further clean up.
          Fix a bug in job submission.

          Show
          Runping Qi added a comment - Further clean up. Fix a bug in job submission.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394668/patch-3770.v2.txt
          against trunk revision 720698.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394668/patch-3770.v2.txt against trunk revision 720698. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3654/console This message is automatically generated.
          Hide
          Runping Qi added a comment -

          Fixed the javadoc warning in the previous version.

          Show
          Runping Qi added a comment - Fixed the javadoc warning in the previous version.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394776/patch-3770.v2.txt
          against trunk revision 721415.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394776/patch-3770.v2.txt against trunk revision 721415. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3669/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          +1

          I just committed this. Thanks, Runping

          Show
          Chris Douglas added a comment - +1 I just committed this. Thanks, Runping
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #677 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/677/ )
          Hide
          yy john added a comment -

          Can anyone help me???

          I am using hadoop-0.20.2 version and try
          to compile gridmix2 under that directory by typing "ant"

          errors showing is

          [javac] Compiling 3 source files to /home/hadoop/src/benchmarks/gridmix2/build
          [javac] /home/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/CombinerJobCreator.java:25: package org.apache.hadoop.fs does not exist
          [javac] import org.apache.hadoop.fs.Path;
          [javac] ^
          [javac] /home/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/CombinerJobCreator.java:26: package org.apache.hadoop.io does not exist
          [javac] import org.apache.hadoop.io.IntWritable;
          ......

          .....

          Can any one help me to build gridmix2 to generate gridmix.jar?

          THANK YOU VERY MUCH!!

          Show
          yy john added a comment - Can anyone help me??? I am using hadoop-0.20.2 version and try to compile gridmix2 under that directory by typing "ant" errors showing is [javac] Compiling 3 source files to /home/hadoop/src/benchmarks/gridmix2/build [javac] /home/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/CombinerJobCreator.java:25: package org.apache.hadoop.fs does not exist [javac] import org.apache.hadoop.fs.Path; [javac] ^ [javac] /home/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/CombinerJobCreator.java:26: package org.apache.hadoop.io does not exist [javac] import org.apache.hadoop.io.IntWritable; ...... ..... Can any one help me to build gridmix2 to generate gridmix.jar? THANK YOU VERY MUCH!!

            People

            • Assignee:
              Runping Qi
              Reporter:
              Lingyun Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development