This META-BUG is here to collect all reports of the "<style> ignores <classpath>" or "<junit> ignores <classpath>" type, which are symptoms of a common root problem. I'll try to describe the problem here and show how to work around the problem, the goal is to gather feedback on this description and add it to the FAQ and the manual after that.
*** Bug 5947 has been marked as a duplicate of this bug. ***
*** Bug 1118 has been marked as a duplicate of this bug. ***
*** Bug 3934 has been marked as a duplicate of this bug. ***
OK, first of all let's state that Ant adds all .jar files from ANT_HOME/lib to CLASSPATH, therefore when I say "in CLASSPATH" in the rest, it means "either in your CLASSPATH environment or ANT_HOME/lib". This bug collects a common type of problem: A task needs an external library and it has a nested classpath element so that you can point it to this external library, but that doesn't work unless you put the external library into the CLASSPATH. The root of the problem is that the class that needs the external library is on the CLASSPATH. When you specify a nested <classpath> in Ant, Ant creates a new class loader that uses the path you have specified. It then tries to load additional classes from this classloader. In most cases - for example the two cases above - Ant doesn't load the external library directly, it is the loaded class that does so. In the case of <junit> it is the task implementation itself and in the case of <style> it is the implementation of the org.apache.tools.ant.taskdefs.XSLTLiaison class. Ant's class loader implementation uses Java's delegation model, see http://java.sun.com/products/jdk/1.2/docs/api/java/lang/ClassLoader.html the paragraph > The ClassLoader class uses a delegation model to search for classes and > resources. Each instance of ClassLoader has an associated parent class loader. > When called upon to find a class or resource, a ClassLoader instance will > delegate the search for the class or resource to its parent class loader > before attempting to find the class or resource itself. The virtual machine's > built-in class loader, called the bootstrap class loader, does not itself have > a parent but may serve as the parent of a ClassLoader instance. This means, Ant's class loader will consult the bootstrap class loader first, which tries to load classes from CLASSPATH. The bootstrap class loader doesn't know anything about Ant's class loader or even the path you have specified. If the bootstrap class loader can load the class Ant has asked it to load, this class will try to load the external library from CLASSPATH as well - it doesn't know anything else - and will not find it unless the library is in CLASSPATH as well. To solve this, you have two major options: (1) put all external libaries you need in CLASSPATH as well this is not what you want, otherwise you wouldn't have found this bug report. (2) remove the class that loads the external library from the CLASSPATH. The easiest way to do this is to remove optional.jar from ANT_HOME. If you do so, you will have to <taskdef> all optional tasks and use nested <classpath> elements in the <taskdef> tasks that point to the new location of optional.jar. Also, don't forget to add the new location of optional.jar to the <classpath> of your <style> or <junit> task. If you want to avoid to <taskdef> all optional tasks you need, the only other option is to remove the classes that should not be loaded via the bootstrap class loader from optional.jar and put them into a separate archive. Add this separate archive to the <classpath> of your <style> or <junit> task - and make sure the separate archive is not in CLASSPATH. In the case of <junit> you'd have to remove all classes that are in the org/apache/tools/ant/taskdefs/optional/junit directory, in the <style> case it is one of the *Liaison classes in org/apache/tools/ant/taskdefs/optional. If you use the option to break up optional.jar for <junit>, you still have to use a <taskdef> with a nested <classpath> to define the junit task. Does this description make any sense? Would it help people who face the same problem you have seen? If not, how can we improve it?
*** Bug 4065 has been marked as a duplicate of this bug. ***
Bug 5947 offers what I think is a better workaround for JUnit, namely the fork=yes option. True, it's not exactly what I want, but neither is it as complex (in my way of thinking) as disassembling and manually reassembling the ant optional framework. I did come across an aside in the docs mentioning that the classpath wouldn't work unless we fork, but better documentation would help with the understanding so folks like me don't spend hours trying to figure out why the classpath tag isn't working in this excellent new build tool that I'm just beginning to figure out :-) A warning from JUnit when it encounters a classpath tag in a non-fork task that wasn't taskdef'ed would also help to bring attention to the problem and help folks with the workaround without resolving it.How to resolve fully? I'll have to cogitate more on that one.
Thanks Jim, yes, fork="true" helps in the case of <junit>, I forgot about it.
*** Bug 5825 has been marked as a duplicate of this bug. ***
Thanks for this very clear description of the problem from ant's point of view. Now let me expose some of the very concrete problems we run because of this bug. We use ant scripts to build our product on various software platform (some linux, some forte/netbeans on windows with the integrated ant). Since our external dependencies shift a lot they are downloaded on the build computer by ant at the beginning of the build process. This is done via xslt analysis of xml descriptors bundled with each of our modules. Installation documentation is also processed with xslt at build time. Putting all libraries in the classpath is therefore not possible since most of the external jars used in a typical build do not exist on the build system before ant start, and if they do they might not have the correct version for the version of the product we are building (moreover the maturity and behaviour of xslt java engines changes a lot from version to version and provider to provider and using the wrong xslt jar with a version of our ant scripts will cause it to fail) Micromanaging ant installations on all the build systems would be a nightmare : some are bundled with other products (forte) and anyway there is a very high risk they would have to be changed on *all* system every time the main build system changed a little or we tried to build an old version. I appreciate this is a difficult problem and I do not expect ant to be fixed at once but I really think it is worth it (and the number of bugs open on this subjets kind of suggests it)
*** Bug 7081 has been marked as a duplicate of this bug. ***
This now is http://jakarta.apache.org/ant/faq.html#delegating-classloader
Stefan, your suggested solution "remove the class that loads the external library from the CLASSPATH" is not sufficient. It fails, for example, in the case of Mappers. The problem is quite the same: The regexp packages are not found, unless they are in lib. And, of course, I cannot remove the Mapper classes from the classpath. There has to be another solution for the problem. For example, TomCat had quite the same thing with JAR's in WEB-INF/lib. For example, if I have a WEB-INF/classes/jndi.properties then it wasn't found before TomCat 3.2.4. I had to put it into TomCat's system class path. Nowadays this works.
I have a suggestion, which fixes most of the problems. More precise, it someone depends on the "right" behaviour of the software components in use. But the most important software component is, of course, Ant. First, let me explain my view of the problem: Any Java class has an associated ClassLoader. The ClassLoader has loaded the class, thus it is able to load other classes from the same JAR file (or files). If two classes A and B have different ClassLoaders and B tries to load a class from A's JAR file with Class.forName(...), then this doesn't necessarily work. This is exactly the problem with Ant. For example, if I load a class MyTask with <taskdef name="mytask" ...><classpath>...</classpath></taskdef> then the class MyTask has the right ClassLoader, but the Ant classes don't. So, if for example the MyTask uses a class MyType implementing a nested element of MyTask, then the IntrospectionHelper will do a Class.forName("MyType"). This fails, because Class.forName uses the ClassLoader of the IntrospectionHelper. The proposed solution works as follows: - Never use Class.forName(). Replace it with a helper method like public static Class myForName(String name) throws ClassNotFoundException { try { return Class.forName(name); } catch (ClassNotFoundException e) { ClassLoader cl = Thread.currentThread().getContextClassLoader(); if (cl == null) { throw e; } return cl.loadClass(name); } - Make sure, that the threads ContextClassLoader is right. For example, use a helper class ThreadContextClassLoader, which grabs the current ContextClassLoader and an additional ClassLoader and uses Class.forName(), original thread ContextClassLoader and the additional ClassLoader, in that order and enclose the perform() method with Task task; ThreadClassLoader tcl = new ThreadClassLoader(task.getClass().getClassLoader()); ClassLoader cl = Thread.currentThread().getClassLoader(); try { Thread.currentThread.setContextClassLoader(tcl); task.perform; } finally { Thread.currentThread.setContextClassLoader(cl); } I offer to create a patch, if someone (Stefan?) discusses details with me and advices me, which version to patch exactly. Regards, Jochen
One problem with you approach is, that the context classloader is not available in JDK 1.1. We already have at least half of the infrastructure for your proposal in place, i.e. AntClassLoader knows how to set and unset itself as the context classloader (via reflection) and the oat.ant.util.LoaderUtil class can retrieve the context class loader via reflection. All that we'd need now: (1) set the context class loader in the right places (2) use it consistently (3) ensure we don't break backwards compatibility I'll be happy to assist here, the context would be no earlier than Ant 1.6 of course.
The statement below is not correct. > then the class MyTask has the right ClassLoader, but the Ant > classes don't. So, if for example the MyTask uses a class MyType > implementing a nested element of MyTask, then the IntrospectionHelper > will do a Class.forName("MyType"). This fails, because Class.forName > uses the ClassLoader of the IntrospectionHelper. Introspection Helper gets the class of the nested type from the class being introspected through reflection. This class will have been loaded by MyTask's loader when the MyTask class was resolved by the VM. Class.forName is not used in IntrospectionHelper unless you are trying to contruct a Class argument which would be very uncommon (probably never). Just FYI...
*** Bug 9841 has been marked as a duplicate of this bug. ***
*** Bug 10931 has been marked as a duplicate of this bug. ***
*** Bug 14931 has been marked as a duplicate of this bug. ***
*** Bug 22170 has been marked as a duplicate of this bug. ***
Since my bug report (22170) has been marked as a duplicate of this one, I guess its better that I put my comment in this instead. I think I might know what the problem is. Since reporting my bug I have made my own ClassLoader for my project, and I got exactly the same exception as Ant does! It complains that it cannot find "Path.class". What I didn't notice at first was that the class it complains about not finding is not having any package! In my case the Path class does belong to a package. But somewhere in my code there is a if (this.interfaceClass.getPackage() != null) ... getPackage() did actually return null when this.interfaceClass had been loaded by my class loader causing "Path.class" without a package to be looked for a bit down using Class.forName(). This ofcourse failed with the ClassNotFoundException. What I did was to add a call to definePackage(...) with the class package before calling defineClass() in findClass(). When the class is then defined the package the class file refers to is also defined and the defined class can reference it. After this fix my class loader works perfectly. If I bring up AntClassLoader.java in my editor and do a find on "definePackage" nothing is found! So AntClassLoader never calls definePackage()! That is the source of my problem! Best Regards, Tommy Svensson
*** Bug 33440 has been marked as a duplicate of this bug. ***
I have a patch that makes junit work without any changes to the ant installation. The only requirement is that you fork a new JVM for the junit test. How did I do this? I changed the ant-junit.jar so it does not load/reference any junit classes before it forks a new JVM. That way the classloader problems disappear since junit classes are not loaded inside ant's JVM. If you want to give it a testspin, get the jar file here: 1) Download ant-junit.jar from here http://tobiasen.dk/ant/ant-junit.jar 2) Drop it in $ANT_HOME/lib 3) Run your tests, remember to set fork="yes" in your tests and have the junit.jar in the classpath of the test. <junit fork="yes" printsummary="yes"> <classpath> include junit.jar here... </classpath> ... I have compiled the ant-junit.jar using jdk 1.4.2_08 and against ant 1.6.4. I would like to submit the patch, but I have no experience in submitting patches. Please tell me how. The patch is quite big, since almost all classes in org/apache/tools/ant/taskdefs/optional/junit are changed and I have introduced some new interfaces.
http://ant.apache.org/ant_task_guidelines.html to read about how to create patches ....
(In reply to comment #22) > I changed the ant-junit.jar so it does not load/reference any > junit classes before it forks a new JVM. That way the classloader problems > disappear since junit classes are not loaded inside ant's JVM. This could be quite useful. E.g. the NetBeans IDE uses http://www.netbeans.org/download/dev/javadoc/org-apache-tools-ant-module/org/apache/tools/ant/module/spi/AutomaticExtraClasspathProvider.html from the module bundling junit.jar in order to force it to be added to Ant's primary classpath. Even though the build scripts that the IDE generates will have an explicit path for junit.jar - which is used e.g. when compiling unit tests - it cannot currently run <junit> unless this has been done. A patch such as is described here would make it possible to do omit junit.jar from Ant's primary classpath, and would mean that a command-line build incl. unit testing would succeed so long as the path was set up correctly, without requiring the user to run Ant with junit.jar in $CLASSPATH (or ant/lib/ or whatever).
Why not simply remove the optional ant tasks from the classpath in the default install of ant? This seems pretty reasonable: * For those that don't mind modifying their ant home directory, it's not really any extra work: since you can't use tasks like <junit> without manually adding junit.jar to the ant lib folder anyway, it doesn't seem like much more work to drag in two jars (ant-junit.jar & junit.jar) rather than one (junit.jar) into the lib folder. * For this (like me) who want to create a user-friendly build process that does not require users to modify their ant installs, I can redistribute the relevant jars with my project and load them within my build.xml file. I want my users to just be able to download my package and type 'ant' to have everything work. The only way I see to do this right now is to copy the source for the JUnit optional task into my own package, so that it is not loaded by the core classloader. This is a pretty crummy workaround.
*** Bug 36862 has been marked as a duplicate of this bug. ***
(In reply to comment #25) > Why not simply remove the optional ant tasks from the classpath in the default install of ant? This > seems pretty reasonable: > > * For those that don't mind modifying their ant home directory, it's not really any extra work: since you > can't use tasks like <junit> without manually adding junit.jar to the ant lib folder anyway, it doesn't > seem like much more work to drag in two jars (ant-junit.jar & junit.jar) rather than one (junit.jar) into > the lib folder. > > * For this (like me) who want to create a user-friendly build process that does not require users to > modify their ant installs, I can redistribute the relevant jars with my project and load them within my > build.xml file. > > I want my users to just be able to download my package and type 'ant' to have everything work. The > only way I see to do this right now is to copy the source for the JUnit optional task into my own > package, so that it is not loaded by the core classloader. This is a pretty crummy workaround. I think this is the best all-around solution to the problem, with one addition: Put the optional ant tasks into a separate folder (optional/lib, opt/lib or some such) and then add a default property called ant.optional.library.dir that is similar to the ant.library.dir except that it points to this new optional library directory. This way, if you want to use an optional task you can simply include the external jars in your distribution and then write a taskdef whose classpath includes the appropriate optional.jar file and its dependencies. While this does create a little more work for anyone wanting to use the optional tasks, the volume of help requests would be simplified by adding an sample taskdef to the doc page of each optional task. As it stands right now, the only workable solution to this problem makes ant much less useful to people trying to build code rather than people trying to write ant scripts. I would imagine that usability should focus first on people trying to build code before it focuses on people trying to write ant scripts because (typically) the person writing the ant script will be somewhat more technical than the other. Besides, it makes the ant script more self documenting with respect to its dependencies.
> this does create a little more work for anyone wanting to use the optional tasks, i.e. it would break every single build file on the planet that uses any optional task. Sam, we cannot do that. we cannot break every build file. We need a better solution. For reference, my project's build process is documented here: http://cvs.sourceforge.net/viewcvs.py/*checkout*/smartfrog/core/antbuild/doc/third_generation_build_process.sxw We have a directory of libraries under SCM, this directory contains the releases of add on tasks that we require for the build. You need either to declare this with a -lib option in ANT_OPTS, or pull the files into ANT_HOME/lib ${user.home}/.ant/lib Life would be simpler if we could somehow declare in a per-project basis an extra dir of stuff to load. All that stuff would need to be unloaded at the end of the build of course, or you have just contaminated a hosting JVM. I just dont see an easy way of doing this.
> Sam, we cannot do that. we cannot break every build file. We need a better > solution. I want to stress that there is a _simple_ solution for this problem. No need to break any existing build files or rearrange jar files. This is true for the junit task anyway. How? Simply fork junit tests into a separate JVM and make sure only the new JVM uses junit classes. There is really no need for the JUnitTask.java task to import junit.framework.Test, it is just sloppy code. I have made this change and it can be done in less than a day! The only drawback is that people that do not fork tests into a separate JVM still needs the current workaround. @See http://issues.apache.org/bugzilla/show_bug.cgi?id=6606#c22
I have filed a separate bug #38799 to cover the specific case of junit.jar in <junit>, since this metabug seems too broad to close directly.
I've got two cross references, both patches available. Bug #47003 makes AntClassLoader in combination with the <classloader> task much more powerful, by ensuring that the ant core class loader will always be an AntClassLoader, which in turn allows addition of class path elements. The result is that you can simply use <classloader> to extend the core ant classpath at need, and don't have to worry about a complicated classloader hierarchy. Bug #47002 takes yet another shot at <junit> and the missing TraXLiaison. This time without forking, but instead through yet another nested <classpath> element. Unfortunately both patches have seen no comments or review yet. Still waiting...