Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1262

Eclipse Plugin does not build for Hadoop 0.20.1

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.20.1, 0.20.2, 0.21.0, 0.22.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      SLES 10, Mac OS/X 10.5.8

    • Release Note:
      This resolves the open issues with the Hadoop eclipse plugin for Eclipse 3.4 and 3.5

      Description

      When trying to run the build script for the Eclipse Plugin in src/contrib/eclipse-plugin there are several errors a user receives. The first error is that the eclipse.home is not set. This is easily remedied by adding a value for eclipse.home in the build.properties file in the eclipse-plugin directory.

      The script then states it cannot compile org.apache.hadoop.eclipse.launch.HadoopApplicationLaunchShortcut because it cannot resolve JavaApplicationLaunchShortcut on line 35:
      import org.eclipse.jdt.internal.debug.ui.launcher.JavaApplicationLaunchShortcut;

      and fails

      I believe this is because there is no jar in the eclipse.home/plugins that has this class in that package. I did however find it in org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut which was inside in org.eclipse.jdt.debug.ui_3.4.1.v20090811_r351.jar in the plugins dir of Eclipse 3.5

      Changing the import in the class in the source to the latter allows the build to complete successfully. The M/R Perspective opens and works on my SLES 10 Linux environment but not on my Macbook Pro. Both are running Eclipse 3.5.

      To users wanting to do the same, I built this inside Eclipse. To do that I added org.eclipse.jdt.debug.ui_3.4.1.v20090811_r351.jar and hadoop-0.20.1-core.jar to the ant runtime configuration classpath. I also had to set the version value=0.20.1 in the build.properties. You will also need to copy hadoop-0.20.1-core.jar to hadoop.home/build and commons-cli-1.2.jar to hadoop.home/build/ivy/lib/Hadoop/common.

      1. hadoop-0.20.1-eclipse-plugin.jar
        2.76 MB
        Stephen Watt
      2. HADOOP-6360.patch
        1 kB
        Stephen Watt

        Issue Links

          Activity

          Hide
          Stephen Watt added a comment -

          This is a compiled eclipse-plugin for Hadoop 0.20.1. Tested on Eclipse 3.5 on Linux.

          Show
          Stephen Watt added a comment - This is a compiled eclipse-plugin for Hadoop 0.20.1. Tested on Eclipse 3.5 on Linux.
          Hide
          steve added a comment -

          Plugin has been working for me, but now all of a sudden I have no Hadoop options as if the plugin never existed and my HDFS connections are gone?
          Plugin is still sitting in my eclipse_home...
          When I look in the "Usage Data Collector --> Preview" I see a bunch of these errors: "Could not acquire children from extension: org.apache.hadoop.eclipse.views.dfscontent".
          Any ideas how to fix this?

          Show
          steve added a comment - Plugin has been working for me, but now all of a sudden I have no Hadoop options as if the plugin never existed and my HDFS connections are gone? Plugin is still sitting in my eclipse_home... When I look in the "Usage Data Collector --> Preview" I see a bunch of these errors: "Could not acquire children from extension: org.apache.hadoop.eclipse.views.dfscontent". Any ideas how to fix this?
          Hide
          Stephen Watt added a comment -

          This is a patch that resolves the issue at both the build and runtime.

          Show
          Stephen Watt added a comment - This is a patch that resolves the issue at both the build and runtime.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12425381/hadoop-0.20.1-eclipse-plugin.jar
          against trunk revision 881509.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/143/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425381/hadoop-0.20.1-eclipse-plugin.jar against trunk revision 881509. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/143/console This message is automatically generated.
          Hide
          Stephen Watt added a comment -

          Modified patch - hopefully QA will accept this

          Show
          Stephen Watt added a comment - Modified patch - hopefully QA will accept this
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12425394/HADOOP-6338.patch
          against trunk revision 881509.

          -1 @author. The patch appears to contain 1 @author tags which the Hadoop community has agreed to not allow in code contributions.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/144/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425394/HADOOP-6338.patch against trunk revision 881509. -1 @author. The patch appears to contain 1 @author tags which the Hadoop community has agreed to not allow in code contributions. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/144/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -
          • Please remove the attribution, per contribution guidelines
            +    /* @author Steve Watt - swatt AT us.ibm.com
            +     * https://issues.apache.org/jira/browse/HADOOP-6360
            +     */
            
          • Since this is proposed for the 0.20 branch and Hudson (automated testing) will attempt to apply it to trunk, the test-patch target must be run manually. Directions are in the prenominate wiki
          Show
          Chris Douglas added a comment - Please remove the attribution, per contribution guidelines + /* @author Steve Watt - swatt AT us.ibm.com + * https://issues.apache.org/jira/browse/HADOOP-6360 + */ Since this is proposed for the 0.20 branch and Hudson (automated testing) will attempt to apply it to trunk, the test-patch target must be run manually. Directions are in the prenominate wiki
          Hide
          Stephen Watt added a comment -

          Hi Chris

          I've removed the @author statement and have subsequently found the HowToContribute WikiPage, so I am in the process of testing it locally to confirm the patch will be accepted before I move this JIRA into Patch Available again.

          Show
          Stephen Watt added a comment - Hi Chris I've removed the @author statement and have subsequently found the HowToContribute WikiPage, so I am in the process of testing it locally to confirm the patch will be accepted before I move this JIRA into Patch Available again.
          Hide
          Stephen Watt added a comment -

          Moving to "Patch Available" - There is no supporting test as this resolves an issue with the build. The Build itself is the test.

          Show
          Stephen Watt added a comment - Moving to "Patch Available" - There is no supporting test as this resolves an issue with the build. The Build itself is the test.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12426690/HADOOP-6360.patch
          against trunk revision 887135.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The patch appears to cause tar ant target to fail.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12426690/HADOOP-6360.patch against trunk revision 887135. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/292/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          The patch causes the build to fail, specifically:

               [exec] compile:
               [exec]      [echo] contrib: eclipse-plugin 
               [exec]     [javac] Compiling 45 source files to /grid/0/hudson/hudson-slave/workspace/Mapreduce-Patch-h6.grid.sp2.yahoo.net/trunk\
                      /build/contrib/eclipse-plugin/classes
               [exec]     [javac] /grid/0/hudson/hudson-slave/workspace/Mapreduce-Patch-h6.grid.sp2.yahoo.net/trunk/src/contrib/eclipse-plugin\
                      /src/java/org/apache/hadoop/eclipse/launch/HadoopApplicationLaunchShortcut.java:35: cannot find symbol
               [exec]     [javac] symbol  : class JavaApplicationLaunchShortcut
               [exec]     [javac] location: package org.eclipse.jdt.debug.ui.launchConfigurations
               [exec]     [javac] import org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut;
          
          Show
          Chris Douglas added a comment - The patch causes the build to fail , specifically: [exec] compile: [exec] [echo] contrib: eclipse-plugin [exec] [javac] Compiling 45 source files to /grid/0/hudson/hudson-slave/workspace/Mapreduce-Patch-h6.grid.sp2.yahoo.net/trunk\ /build/contrib/eclipse-plugin/classes [exec] [javac] /grid/0/hudson/hudson-slave/workspace/Mapreduce-Patch-h6.grid.sp2.yahoo.net/trunk/src/contrib/eclipse-plugin\ /src/java/org/apache/hadoop/eclipse/launch/HadoopApplicationLaunchShortcut.java:35: cannot find symbol [exec] [javac] symbol : class JavaApplicationLaunchShortcut [exec] [javac] location: package org.eclipse.jdt.debug.ui.launchConfigurations [exec] [javac] import org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut;
          Hide
          Stephen Watt added a comment -

          This failure is a little weird as I get no such error when I run the apply-patch locally against the MapReduce project trunk. The only -1 I get is for lack of contrib tests, everything else is +1.

          The only thing I can think of is the eclipse.home that Hudson is using is using a version of Eclipse older than 3.4. With Eclipse 3.4 and 3.5, the plugins folder contains org.eclipse.jdt.debug.ui_3.3.1.v20080730_r341.jar which contains org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut.

          If the file exists, it is specifically included in the src/contrib/eclipse-plugin/build.xml script and should therefore work, as you can see below :

          <project default="jar" name="eclipse-plugin">

          <import file="../build-contrib.xml"/>

          <path id="eclipse-sdk-jars">
          <fileset dir="$

          {eclipse.home}

          /plugins/">
          <include name="org.eclipse.ui*.jar"/>
          <include name="org.eclipse.jdt*.jar"/>
          <include name="org.eclipse.core*.jar"/>
          <include name="org.eclipse.equinox*.jar"/>
          <include name="org.eclipse.debug*.jar"/>
          <include name="org.eclipse.osgi*.jar"/>
          <include name="org.eclipse.swt*.jar"/>
          <include name="org.eclipse.jface*.jar"/>

          <include name="org.eclipse.team.cvs.ssh2*.jar"/>
          <include name="com.jcraft.jsch*.jar"/>
          </fileset>
          </path>

          Show
          Stephen Watt added a comment - This failure is a little weird as I get no such error when I run the apply-patch locally against the MapReduce project trunk. The only -1 I get is for lack of contrib tests, everything else is +1. The only thing I can think of is the eclipse.home that Hudson is using is using a version of Eclipse older than 3.4. With Eclipse 3.4 and 3.5, the plugins folder contains org.eclipse.jdt.debug.ui_3.3.1.v20080730_r341.jar which contains org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut. If the file exists, it is specifically included in the src/contrib/eclipse-plugin/build.xml script and should therefore work, as you can see below : <project default="jar" name="eclipse-plugin"> <import file="../build-contrib.xml"/> <path id="eclipse-sdk-jars"> <fileset dir="$ {eclipse.home} /plugins/"> <include name="org.eclipse.ui*.jar"/> <include name="org.eclipse.jdt*.jar"/> <include name="org.eclipse.core*.jar"/> <include name="org.eclipse.equinox*.jar"/> <include name="org.eclipse.debug*.jar"/> <include name="org.eclipse.osgi*.jar"/> <include name="org.eclipse.swt*.jar"/> <include name="org.eclipse.jface*.jar"/> <include name="org.eclipse.team.cvs.ssh2*.jar"/> <include name="com.jcraft.jsch*.jar"/> </fileset> </path>
          Hide
          Eli Collins added a comment -

          The patch on MAPREDUCE-1280 covers this.

          Show
          Eli Collins added a comment - The patch on MAPREDUCE-1280 covers this.
          Hide
          Michael Aro added a comment -

          Improving the Eclipse Plugin "http://wiki.apache.org/hadoop/ProjectSuggestions". What is the latest status and the next steps in case improvement is needed still?

          Show
          Michael Aro added a comment - Improving the Eclipse Plugin "http://wiki.apache.org/hadoop/ProjectSuggestions". What is the latest status and the next steps in case improvement is needed still?

            People

            • Assignee:
              Unassigned
              Reporter:
              Stephen Watt
            • Votes:
              3 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development