Hadoop Common
  1. Hadoop Common
  2. HADOOP-6407

Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently Eclipse configuration (namely .classpath) isn't synchronized automatically when lib versions are changed. This causes great inconvenience so people have to change their project settings manually, etc.

      It'd be great if these configs could be updated automatically every time such a change takes place, e.g. whenever ivy is pulling in new version of a jar.

      1. HADOOP-6407.patch
        11 kB
        Tom White
      2. HADOOP-6407.patch
        11 kB
        Tom White
      3. HADOOP-6407.patch
        11 kB
        Tom White

        Issue Links

          Activity

          Hide
          Konstantin Boudnik added a comment -

          This modification needs to be done for all subprojects of Hadoop.

          Show
          Konstantin Boudnik added a comment - This modification needs to be done for all subprojects of Hadoop.
          Hide
          Konstantin Boudnik added a comment -

          It seems like an easy way to solve this is to

          • modify .eclipse.templates/.classpath to include only source code folders and some permanent entries like build/classes. All dynamic content has to be replaces with <classpathentry kind="lib" path="build/ivy/lib/**

          Then a separate process invoked by ant build will expand the macro def. above to the actual content of the directory in question whenever a change of the libs occurred. Even ant eclipse-files can be executed automatically after that. Then the user will simply need to refresh the project in Eclipse IDE.

          Show
          Konstantin Boudnik added a comment - It seems like an easy way to solve this is to modify .eclipse.templates/.classpath to include only source code folders and some permanent entries like build/classes . All dynamic content has to be replaces with <classpathentry kind="lib" path="build/ivy/lib/** Then a separate process invoked by ant build will expand the macro def. above to the actual content of the directory in question whenever a change of the libs occurred. Even ant eclipse-files can be executed automatically after that. Then the user will simply need to refresh the project in Eclipse IDE.
          Hide
          Tom White added a comment -

          Another way to do it would be to use Ant-Eclipse (http://sourceforge.net/projects/ant-eclipse/) which generates Eclipse files from Ivy files. Avro and ZooKeeper take this approach (AVRO-146, ZOOKEEPER-539).

          Show
          Tom White added a comment - Another way to do it would be to use Ant-Eclipse ( http://sourceforge.net/projects/ant-eclipse/ ) which generates Eclipse files from Ivy files. Avro and ZooKeeper take this approach ( AVRO-146 , ZOOKEEPER-539 ).
          Hide
          Konstantin Boudnik added a comment -

          That perhaps even better! Thanks for the pointer.

          Show
          Konstantin Boudnik added a comment - That perhaps even better! Thanks for the pointer.
          Hide
          Tom White added a comment -

          Here's a patch for review that adds an "eclipse" target (like the one in Avro and ZooKeeper) that generates the .classpath and .project files for Eclipse using Ant-Eclipse. Note that it doesn't keep the Hadoop Ant builder that ran the code generator for RecordIO and Avro records: this is easy (and rare) enough to do from the command line with "ant compile-core-test".

          I've tested this on Eclipse 3.4.1.

          Show
          Tom White added a comment - Here's a patch for review that adds an "eclipse" target (like the one in Avro and ZooKeeper) that generates the .classpath and .project files for Eclipse using Ant-Eclipse. Note that it doesn't keep the Hadoop Ant builder that ran the code generator for RecordIO and Avro records: this is easy (and rare) enough to do from the command line with "ant compile-core-test". I've tested this on Eclipse 3.4.1.
          Hide
          Karthik K added a comment -

          Just a piece of suggestion - can that task be refactored appropriately into a separate file - eclipse-targets.xml with comments mentioning the variables/properties that need to be set for the same. Since this would useful for a number of projects and we can copy the file as such.

          Show
          Karthik K added a comment - Just a piece of suggestion - can that task be refactored appropriately into a separate file - eclipse-targets.xml with comments mentioning the variables/properties that need to be set for the same. Since this would useful for a number of projects and we can copy the file as such.
          Hide
          Konstantin Boudnik added a comment -

          I've updated the JIRA to indicate that 0.21 is also has this problem.

          Show
          Konstantin Boudnik added a comment - I've updated the JIRA to indicate that 0.21 is also has this problem.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429184/HADOOP-6407.patch
          against trunk revision 895831.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429184/HADOOP-6407.patch against trunk revision 895831. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/251/console This message is automatically generated.
          Hide
          Tom White added a comment -

          Minor change to correct the deletion of the downloaded Ant-Eclipse package. Also, I tested this with Eclipse Galileo. After generating the Eclipse files with "ant eclipse" you should clean the project from within Eclipse.

          Just a piece of suggestion - can that task be refactored appropriately into a separate file - eclipse-targets.xml with comments mentioning the variables/properties that need to be set for the same.

          I agree it would be good to modularize the build files between projects: this should be a separate set of JIRA issues. For the present issue, there's not a lot of scope for sharing ant code, since the "eclipse" targets, for example, will differ between projects since the source paths are different (different contrib projects).

          Show
          Tom White added a comment - Minor change to correct the deletion of the downloaded Ant-Eclipse package. Also, I tested this with Eclipse Galileo. After generating the Eclipse files with "ant eclipse" you should clean the project from within Eclipse. Just a piece of suggestion - can that task be refactored appropriately into a separate file - eclipse-targets.xml with comments mentioning the variables/properties that need to be set for the same. I agree it would be good to modularize the build files between projects: this should be a separate set of JIRA issues. For the present issue, there's not a lot of scope for sharing ant code, since the "eclipse" targets, for example, will differ between projects since the source paths are different (different contrib projects).
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429477/HADOOP-6407.patch
          against trunk revision 895831.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429477/HADOOP-6407.patch against trunk revision 895831. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/254/console This message is automatically generated.
          Hide
          Tom White added a comment -

          I'd like to commit this in the next day or so unless there are any objections.

          Show
          Tom White added a comment - I'd like to commit this in the next day or so unless there are any objections.
          Hide
          Konstantin Boudnik added a comment -

          +1 patch looks good.

          Committing this sounds like a good idea to me. Do you plan to propagate the change to MR/HDFS?

          Show
          Konstantin Boudnik added a comment - +1 patch looks good. Committing this sounds like a good idea to me. Do you plan to propagate the change to MR/HDFS?
          Hide
          Tom White added a comment -

          New patch that applies against trunk. I'll commit it after Hudson has checked it. I've also opened HDFS-1035 and MAPREDUCE-1592 to do the same thing for HDFS and MapReduce.

          Show
          Tom White added a comment - New patch that applies against trunk. I'll commit it after Hudson has checked it. I've also opened HDFS-1035 and MAPREDUCE-1592 to do the same thing for HDFS and MapReduce.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12438531/HADOOP-6407.patch
          against trunk revision 921980.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438531/HADOOP-6407.patch against trunk revision 921980. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/416/console This message is automatically generated.
          Hide
          Tom White added a comment -

          I've just committed this.

          Show
          Tom White added a comment - I've just committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #201 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/201/)
          . Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #201 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/201/ ) . Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #282 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/282/)
          . Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #282 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/282/ ) . Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy.

            People

            • Assignee:
              Tom White
              Reporter:
              Konstantin Boudnik
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development