Bug 38223

Summary:	ConfigurableClasspathWebappLoader
Product:	Tomcat 5	Reporter:	Sriram Narayanan <sriramnrn>
Component:	Catalina	Assignee:	Tomcat Developers Mailing List <dev>
Status:	RESOLVED WONTFIX
Severity:	enhancement
Priority:	P4
Version:	5.5.14
Target Milestone:	---
Hardware:	Other
OS:	other
Attachments:	Files demonstrating how this convenience class might be used Configurable webapp loader that can add to the web application classpath Picks classpath from an XML file

Description Sriram Narayanan 2006-01-11 17:04:09 UTC

This is a convenience class, and not at all compliant with the Servlet Spec.

By using this WebAppLoader, once can add entries to the Web Application's class
path (outside of WEB-INF/classes and WEB-INF/lib).

I created this WebAppLoader in response to
http://marc.theaimsgroup.com/?l=tomcat-user&m=113682917308106&w=2

The context.xml looks like this:
<Context docBase="d:\temp\testing" reloadable="true" >
	<Loader 
		className="com.thoughtworks.tomcat.loader.ConfigurableClasspathWebappLoader"
		classpathEntriesFile="d:\\temp\\myclasses.properties"
	/>
</Context>

The file myclasses.properties contains entries like this:
PathCount=2
Path.1=file:/d:/temp/bin/
Path.2=file:/c:/temp/deleteThis/

Now, classes kept in these two paths can be available to the Web Application.
One need not have the classes copied into the WEB-INF/classes folder.

So, one might have a JSP that contains lines like this :
=========================================================
<%
	Class clazz  = Class.forName("com.thoughtworks.deletable.SomeStuff");
	Object anObject = clazz.newInstance();
	String value = anObject.toString();
		
	com.thoughtworks.deletable.SomeStuff cyber = new
com.thoughtworks.deletable.SomeStuff();
		
	String anotherValue = cyber.toString();
		
%>

The value is <%=value%>

<br/>

Other values can be interesting too <%=anotherValue%>
=========================================================

Comment 1 Sriram Narayanan 2006-01-11 17:34:14 UTC

Created attachment 17383 [details]
Files demonstrating how this convenience class might be used

1. Extract testing.zip at some location.
2. Follow the instructions gives in the readme.txt file that is kept within
this zip.

Comment 2 Oded Arbel 2006-01-12 19:20:44 UTC

Created attachment 17402 [details]
Configurable webapp loader that can add to the web application classpath

Sriram: Thanks for the pointer.  
 
I didn't much like the implementation though - (a) I don't like the need to 
create a properties file: though it will greatly simplify the setup if you 
have multiple web applications with the same requirements, for me this is 
needed for development work only and I don't want developers to get too 
comfortable with this hack, and (b) I didn't like the fact that the attached 
file doesn't have the source for the class loader, so I couldn't see for myself
how it does this and fix my first problem.

So I created a new class, based on WebappLoader (not WebappClassLoader), which
very simply offers a 'classpath' attribute for the loader, which when filled
with classpath entries (colon separated or whatever your platform likes) it
adds them to the list of repositories in WebappLoader.

So my context looks like this:
<Context path="/my/webapp" docBase="/path/to/my/webapp" debug="0"
privileged="true" reloadable="true">
    <Loader
className="org.apache.catalina.loader.ConfigurableWebappClassLoader"
classpath="/path/to/extra/classes:/path/to/more/classes"/>
</Context>

I've tested it to work nicely with Tomcat 5.0.30

I license this code under triple licenses: GPL, LGPL and Apache. I won't mind
if this be added to the Tomcat distribution, though at this point I don't mind
- I only build development servers once in quite a while and the current one is
all done :-)

Comment 3 Sriram Narayanan 2006-01-12 19:36:13 UTC

(In reply to comment #2)
> Created an attachment (id=17402) [edit]
> Configurable webapp loader that can add to the web application classpath
> 
> Sriram: Thanks for the pointer.  
>  
> I didn't much like the implementation though - (a) I don't like the need to 
> create a properties file: though it will greatly simplify the setup if you 
> have multiple web applications with the same requirements, for me this is 
> needed for development work only and I don't want developers to get too 
> comfortable with this hack, 

This is indeed too much a feature for production use. I added the properties
file support because you'd mentioned in your email that you've a large numebr of
paths to add to the CLASSPATH. You'd mentioned fifteen lines, and fifteen lines
worth of CLASSPATH entries are better added individually and separately, than
clubbed together in one large string.

> and (b) I didn't like the fact that the attached 
> file doesn't have the source for the class loader, so I couldn't see for myself
> how it does this and fix my first problem.
> 

The attachment contains a zip file which contains the Eclipse project (complete
with .project and .classpath files). I just checked the attachment and I see
that it contains this particular zip file as well.

So, you have everything in my attachment to see if how I've addressed the
problem. See what you think of the source.

> So I created a new class, based on WebappLoader (not WebappClassLoader), which
> very simply offers a 'classpath' attribute for the loader, which when filled
> with classpath entries (colon separated or whatever your platform likes) it
> adds them to the list of repositories in WebappLoader.
> 
> So my context looks like this:
> <Context path="/my/webapp" docBase="/path/to/my/webapp" debug="0"
> privileged="true" reloadable="true">
>     <Loader
> className="org.apache.catalina.loader.ConfigurableWebappClassLoader"
> classpath="/path/to/extra/classes:/path/to/more/classes"/>
> </Context>
> 
> I've tested it to work nicely with Tomcat 5.0.30
> 
> I license this code under triple licenses: GPL, LGPL and Apache. I won't mind
> if this be added to the Tomcat distribution, though at this point I don't mind
> - I only build development servers once in quite a while and the current one is
> all done :-)
>

Comment 4 Oded Arbel 2006-01-12 20:12:16 UTC

Sorry, I didn't see your sources at first  
   
> I added the properties file support because you'd mentioned in your    
> email that you've a large numebr of paths to add to the CLASSPATH.    
> You'd mentioned fifteen lines, and fifteen lines worth of CLASSPATH    
> entries are better added individually and separately, than clubbed    
> together in one large string.   
   
I agree. The main problem is that I want to add only a small number of   
required projects for each webapp, but have many webapps with different   
requirements, which made the class path quite large as it had to address all   
the requirements in one place.   
   
I agree that clobbering everything in a single classpath element might be a   
problem. Is it possible to have child tags similar to an ant's fileset tag to   
include additional paths each with its own element ? from a quick look at the   
digester's interaction with the loader I couldn't see such a thing.  
 
Your code looks nice - I see you've implemented addRepository() - what is its 
use, except for interface completion ? I saw the bean-info object relate to 
something called "events" which lists 'add' methods.

Comment 5 Sriram Narayanan 2006-01-12 20:35:31 UTC

(In reply to comment #4)
> Sorry, I didn't see your sources at first  

Heh, I guess you were excited to try out the files :) No problems :)

>    
> > I added the properties file support because you'd mentioned in your    
> > email that you've a large numebr of paths to add to the CLASSPATH.    
> > You'd mentioned fifteen lines, and fifteen lines worth of CLASSPATH    
> > entries are better added individually and separately, than clubbed    
> > together in one large string.   
>    
> I agree. The main problem is that I want to add only a small number of   
> required projects for each webapp, but have many webapps with different   
> requirements, which made the class path quite large as it had to address all   
> the requirements in one place.   
>    

Oh... that'd have also caused you problems where only the first set of classes
of two different versions of the same classes in the CLASSPATH would be loaded.

e.g. CLASSPATH=jdom1.0.jar;jdom2.0.jar would cause only jdom1.0.jar to be
searched for the class org.jdom.Dummy which would be present in both the jars.

You've had to contend with a lot of problems all this while !

> I agree that clobbering everything in a single classpath element might be a   
> problem. Is it possible to have child tags similar to an ant's fileset tag to   
> include additional paths each with its own element ? from a quick look at the   
> digester's interaction with the loader I couldn't see such a thing.  
>  
This is possible, but would involve you playing with the
org.apache.catalina.startup.ContextRuleSet class. I'm working on another
approach where the settings are loaded from an XML file. I'll upload this
version once I finish it.

> Your code looks nice - I see you've implemented addRepository() - what is its 
> use, except for interface completion ? I saw the bean-info object relate to 
> something called "events" which lists 'add' methods. 

This is a method that I'd added for tracing purposes. The idea is for users of
this class to try it, and see the debug messages in case they face any problems.

Comment 6 Sriram Narayanan 2006-01-16 19:58:08 UTC

Created attachment 17439 [details]
Picks classpath from an XML file

You can specify the classpath via an XML file that can be placed anywhere on
the file system.

The context.xml contains:
<Context docBase="d:\temp\testing" reloadable="true" >
	<Loader 
	       
className="com.thoughtworks.tomcat.loader.ConfigurableClasspathWebappLoader"
		classpathEntriesFile="d:\\temp\\configurations.xml"
		
	/>
</Context>

The XML structure is as follows:
<configuration>
	<classpaths>
		<path>
			file:/d:/temp/bin/
		</path>
		<path>
			file:/c:/temp/deleteThis/
		</path>
	</classpaths>
</configuration>

Comment 7 Oded Arbel 2006-01-17 11:36:56 UTC

Hi. I've tried to get your latest code to work with my Tomcat 5.0.30. I of
course recompiled the source, as I'm using JDK 1.4.2. 
I also had to change the digester from tomcat.util.digester to commons-digester
as the former package is not available on my installation (tomcat 5.5 only ?).

Now when I load my web application, everytime I try to access something which is
handled by the default servlet (such as images or static html), I get a 404
error - "servlet default is not available".

Apparently, with the configurable web app class loader loaded, tomcat can't find
DefaultServlet from servlets-default.jar. I'm not sure why is that, but with my
implementation it does the same.

I enabled full debug logging for tomcat, and Here are the class loader hierarchy
for a normal web application:

 DEBUG [main] org.apache.commons.modeler.BaseModelMBean - preRegister
WebappClassLoader
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
StandardClassLoader
  delegate: true
  repositories:
    file:/var/lib/tomcat5/server/classes/
    file:/var/lib/tomcat5/server/lib/catalina-i18n-es.jar
    file:/usr/share/java/regexp-1.3.jar
    file:/var/lib/tomcat5/server/lib/tomcat-jk2.jar
    file:/var/lib/tomcat5/server/lib/tomcat-coyote.jar
    file:/var/lib/tomcat5/server/lib/servlets-webdav.jar
    file:/usr/share/java/jakarta-commons-fileupload-1.0.jar
    file:/usr/share/java/mx4j/mx4j-jmx-3.0.1.jar
    file:/var/lib/tomcat5/server/lib/tomcat-util.jar
    file:/var/lib/tomcat5/server/lib/catalina.jar
    file:/var/lib/tomcat5/server/lib/catalina-i18n-fr.jar
    file:/var/lib/tomcat5/server/lib/tomcat-http11.jar
    file:/usr/java/j2sdk1.4.2_09/jre/lib/rt.jar
    file:/usr/share/java/catalina-ant-5.0.30.jar
    file:/usr/share/java/jakarta-commons-digester-1.7.jar
    file:/var/lib/tomcat5/server/lib/servlets-default.jar
    file:/usr/share/java/jakarta-commons-modeler-1.1.jar
    file:/var/lib/tomcat5/server/lib/servlets-invoker.jar
    file:/usr/share/java/jakarta-commons-logging-1.0.4.jar
    file:/var/lib/tomcat5/server/lib/catalina-cluster.jar
    file:/var/lib/tomcat5/server/lib/catalina-i18n-ja.jar
    file:/usr/share/java/jakarta-commons-beanutils-1.7.0.jar
    file:/usr/share/java/jakarta-commons-el-1.0.jar
    file:/var/lib/tomcat5/server/lib/catalina-optional.jar
    file:/var/lib/tomcat5/server/lib/tomcat-jk.jar
    file:/var/lib/tomcat5/server/lib/servlets-common.jar
    file:/var/lib/tomcat5/server/lib/tomcat-jni.jar
----------> Parent Classloader:
StandardClassLoader
  delegate: true
  repositories:
    file:/var/lib/tomcat5/common/classes/
    file:/usr/share/java/xml-commons-apis-1.3.02.jar
    file:/usr/share/java/xerces-j2-2.7.1.jar
    file:/usr/share/java/jta-1.0.1.jar
    file:/usr/java/j2sdk1.4.2_09/jre/lib/rt.jar
    file:/usr/share/java/mx4j/mx4j-jmx-3.0.1.jar
    file:/usr/share/java/jakarta-commons-pool-1.2.jar
    file:/usr/java/j2sdk1.4.2_09/jre/lib/rt.jar
    file:/usr/share/java/jasper5-compiler-5.0.30.jar
    file:/usr/share/java/ant-1.6.2.jar
    file:/usr/share/java/servletapi5-5.0.30.jar
    file:/var/lib/tomcat5/common/lib/naming-java.jar
    file:/var/lib/tomcat5/common/lib/naming-resources.jar
    file:/usr/share/java/jaf-1.0.2.jar
    file:/usr/share/java/ant-launcher-1.6.2.jar
    file:/usr/share/java/jspapi-5.0.30.jar
    file:/usr/share/java/javamail-1.3.3_01.jar
    file:/usr/share/java/jakarta-commons-el-1.0.jar
    file:/usr/share/java/jakarta-commons-collections-3.1.jar
    file:/usr/share/java/jakarta-commons-logging-api-1.0.4.jar
    file:/var/lib/tomcat5/common/lib/naming-factory.jar
    file:/var/lib/tomcat5/common/lib/naming-common.jar
    file:/usr/share/java/jasper5-runtime-5.0.30.jar
    file:/usr/share/java/jakarta-commons-dbcp-1.2.1.jar
----------> Parent Classloader:
sun.misc.Launcher$AppClassLoader@7b7072

And here is the same output for a web application loaded with your configurable
class loader implementation (the output looks identical to the one with my
implementation, I think)

DEBUG [main] org.apache.commons.modeler.BaseModelMBean - preRegister
WebappClassLoader
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
StandardClassLoader
  delegate: true
  repositories:
    file:/var/lib/tomcat5/shared/classes/
----------> Parent Classloader:
StandardClassLoader
  delegate: true
  repositories:
    file:/var/lib/tomcat5/common/classes/
    file:/usr/share/java/xml-commons-apis-1.3.02.jar
    file:/usr/share/java/xerces-j2-2.7.1.jar
    file:/usr/share/java/jta-1.0.1.jar
    file:/usr/java/j2sdk1.4.2_09/jre/lib/rt.jar
    file:/usr/share/java/mx4j/mx4j-jmx-3.0.1.jar
    file:/usr/share/java/jakarta-commons-pool-1.2.jar
    file:/usr/java/j2sdk1.4.2_09/jre/lib/rt.jar
    file:/usr/share/java/jasper5-compiler-5.0.30.jar
    file:/usr/share/java/ant-1.6.2.jar
    file:/usr/share/java/servletapi5-5.0.30.jar
    file:/var/lib/tomcat5/common/lib/naming-java.jar
    file:/var/lib/tomcat5/common/lib/naming-resources.jar
    file:/usr/share/java/jaf-1.0.2.jar
    file:/usr/share/java/ant-launcher-1.6.2.jar
    file:/usr/share/java/jspapi-5.0.30.jar
    file:/usr/share/java/javamail-1.3.3_01.jar
    file:/usr/share/java/jakarta-commons-el-1.0.jar
    file:/usr/share/java/jakarta-commons-collections-3.1.jar
    file:/usr/share/java/jakarta-commons-logging-api-1.0.4.jar
    file:/var/lib/tomcat5/common/lib/naming-factory.jar
    file:/var/lib/tomcat5/common/lib/naming-common.jar
    file:/usr/share/java/jasper5-runtime-5.0.30.jar
    file:/usr/share/java/jakarta-commons-dbcp-1.2.1.jar
----------> Parent Classloader:
sun.misc.Launcher$AppClassLoader@7b7072

As you can see, for some reason one level of StandardClassLoader, which contains
the servlets-default.jar is missing for some reason. 

Can you guess what's going on ?

Comment 8 Sriram Narayanan 2006-01-17 16:56:12 UTC

(In reply to comment #7)
> Hi. I've tried to get your latest code to work with my Tomcat 5.0.30. I of
> course recompiled the source, as I'm using JDK 1.4.2. 

Ack. I'll chance the JDK compatibility level in my Eclipse IDE. And I'm testing
against Tomcat 5.5.15 Beta.

> I also had to change the digester from tomcat.util.digester to commons-digester
> as the former package is not available on my installation (tomcat 5.5 only ?).
> 

Ack. This is interesting. Tomcat 5.5.15 has tomcat.util.digester instead of
commons-digester. I don't know why this is the reason - a search in the archives
should yield some answer, and I'll try to find out this answer.

> Now when I load my web application, everytime I try to access something which is
> handled by the default servlet (such as images or static html), I get a 404
> error - "servlet default is not available".
> 

Ack. I just tested this on both Tomcat 5.5.15 as well as 5.0.30. I'm able to run
the "testing" and the documentation web apps. I'm also able to view the images
and HTML from within these apps on both the Tomcat versions.

> Apparently, with the configurable web app class loader loaded, tomcat can't find
> DefaultServlet from servlets-default.jar. I'm not sure why is that, but with my
> implementation it does the same.
> 
> I enabled full debug logging for tomcat, and Here are the class loader hierarchy
> for a normal web application:
> 
<snip/>
> 
> As you can see, for some reason one level of StandardClassLoader, which contains
> the servlets-default.jar is missing for some reason. 
> 
> Can you guess what's going on ?

I've to leave for the day, and will look into this tomorrow.

-- Sriram

Comment 9 Mathieu 2006-03-07 11:02:22 UTC

(In reply to comment #8)
> (In reply to comment #7)
> > Hi. I've tried to get your latest code to work with my Tomcat 5.0.30. I of
> > course recompiled the source, as I'm using JDK 1.4.2. 
> 
> Ack. I'll chance the JDK compatibility level in my Eclipse IDE. And I'm testing
> against Tomcat 5.5.15 Beta.
> 
> > I also had to change the digester from tomcat.util.digester to commons-digester
> > as the former package is not available on my installation (tomcat 5.5 only ?).
> > 
> 
> Ack. This is interesting. Tomcat 5.5.15 has tomcat.util.digester instead of
> commons-digester. I don't know why this is the reason - a search in the archives
> should yield some answer, and I'll try to find out this answer.
> 
> > Now when I load my web application, everytime I try to access something which is
> > handled by the default servlet (such as images or static html), I get a 404
> > error - "servlet default is not available".
> > 
> 
> Ack. I just tested this on both Tomcat 5.5.15 as well as 5.0.30. I'm able to run
> the "testing" and the documentation web apps. I'm also able to view the images
> and HTML from within these apps on both the Tomcat versions.
> 
> > Apparently, with the configurable web app class loader loaded, tomcat can't find
> > DefaultServlet from servlets-default.jar. I'm not sure why is that, but with my
> > implementation it does the same.
> > 
> > I enabled full debug logging for tomcat, and Here are the class loader hierarchy
> > for a normal web application:
> > 
> <snip/>
> > 
> > As you can see, for some reason one level of StandardClassLoader, which contains
> > the servlets-default.jar is missing for some reason. 
> > 
> > Can you guess what's going on ?
> 
> I've to leave for the day, and will look into this tomorrow.
> 
> -- Sriram

Comment 10 Yoav Shapira 2006-04-14 15:28:01 UTC

Sriram: thanks for contributing this.  It's an interesting idea and I can see
how it'd be useful to some specific applications.  However, I don't see it as
generally necessary and I don't like that it's not Spec-compliant.  We already
have some configurtability around the classloaders and their repositories (see
the conf/catalina.properties file that ships with Tomcat), so I think this
feature, while nice, doesn't have a place in the Tomcat core distribution. 
Accordingly, I'm closing this as WONTFIX.  Of course, if other Tomcat developers
are interested in adding this, we can reopen this issue and/or discuss it on the
dev mailing list.

Oded and anyone else interested: feel free to continue developing, using, and
discussing this on the Tomcat user mailing lists or any other forum of your choice.

Comment 11 Matthew Firth 2006-04-19 09:17:05 UTC

FYI, something similar to this exists in the "DevLoader" component of the 
Sysdeo Tomcat Plugin for Eclipse.

http://www.sysdeo.com/eclipse/tomcatplugin