Bug 51294 - Since 7.0.12 do not work option unpackWARs=true for WARs outside appBase
Summary: Since 7.0.12 do not work option unpackWARs=true for WARs outside appBase
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 7.0.12
Hardware: All All
: P2 enhancement with 7 votes (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-30 13:04 UTC by Andrey M
Modified: 2021-05-28 09:30 UTC (History)
7 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey M 2011-05-30 13:04:34 UTC
In 7.0.12 was added in Changelog this line:
--
Don't unpack WAR files if they are not located in the Host's appBase. (markt)
--
It's change method of deployment from older versions of Tomcat (5.5, 6.0, 7.0.11) and forced to change logics on early configured servers.

May be need to add new parameter like 'unpackExternalWARs=true' to be able to configure this? (Why outside WARs is worse than inside?)
Comment 1 Mark Thomas 2011-05-31 08:30:13 UTC
The documented and intended behaviour of unpackWARs is that it only applies to WARs in the appBase.  This is true for 5.5.x, 6.0.x and 7.0.x.

Unpacking WARs from outside the appBase into the appBase should be possible but adds complexity to the autoDeploy process that I am not convinced is worth it.

If an app needs to be unpacked there are two options:
- deploy the WAR in the appBase with unpackWARs=true
- deploy outside the appBase using a directory rather than a WAR
Comment 2 Andrey M 2011-05-31 13:19:59 UTC
What about Context Container documentation:

unpackWAR: If true, Tomcat will unpack all compressed web applications before running them. If not specified, the default value is true.

When I use <Context path="/app" docBase="${home}/war/app.war"/> I can't find unpacked WAR content in appBase and see only part of WAR in WORK directory (only libs and something from META-INF)
Comment 3 Mark Thomas 2011-06-02 11:22:40 UTC
The unpackWAR option of the context is so rarely used I had actually forgotten it even existed.

http://tomcat.markmail.org/thread/rk6fw5ooohgiqklu explains why it was introduced. Even then it was as an override to unpackWARs at the host level.

The original intention, the current implementation and the current documentation are all different. For now, I'm going to take the easy way out which is to align the documentation with the current implementation. I'm not against changing the implementation to reflect the original intention but that probably needs some discussion on the users list as to what behaviour makes most sense.

I don't see any of the WAR unpacking options being changed to trigger the unpacking of an external WAR in the appBase without a strong use case that can't be met by alternative configurations.
Comment 4 Mark Thomas 2011-06-02 13:09:55 UTC
Docs updated in 7.0.x and will be included in 7.0.15 onwards.
Comment 5 Andrey M 2011-06-02 17:11:35 UTC
Strange choice method of what must be fixed...
Here [http://webcache.googleusercontent.com/search?q=cache:PnXpxLit9OwJ:osdir.com/ml/users-tomcat.apache.org/2011-04/msg00351.html&cd=4&hl=ru&ct=clnk&gl=ru&source=www.google.ru] You fix sources to be compliant to documentation, in my bug request You fix documentation to be compliant to sources... Where is truth? Truth is out there...
Comment 6 archmisha 2011-08-10 14:14:11 UTC
I have the same problem when upgraded to the new version.

But reviewing the following documentation link:
http://tomcat.apache.org/tomcat-7.0-doc/config/host.html#Automatic%20Application%20Deployment
Section "Automatic Application Deployment" explicitly states: "...The docBase attribute of this <Context> element must only be set if the docBase is outside the Host's appBase...."

What resolution you proposed in our case?
Comment 7 Tiago Cruz 2011-12-20 21:45:01 UTC
I also have this problem with tomcat >= 7.0.12 :/

Is this configuration possible? (unpackExternalWARs=true) :D

'Cause to use with configuration management (puppet, chef e etc) its easier to put the .war up-to-date in one directory and use a Context with one docBase using the full path of application.

Thanks!
Comment 8 Konstantin Kolinko 2012-01-17 16:03:46 UTC
Recent discussion on dev (Re: r1080719):
http://tomcat.markmail.org/thread/q6lfopguhnedtbzt
Comment 9 Philippe Busque 2012-02-02 15:05:37 UTC
This if is a major show stopper. It was working fine before, why 'fix' it?

We are currently investigating into upgrading from Tomcat 6 to Tomcat 7.
We have HUNDREDS of Tomcat application running. Some of require to be able to manipulate the webapps content at runtime.  Due to the amount of instance to manage, we link our war outside the webapp for better management.

This 'fix' break any Tomcat server that require an unpacked war and use any management tools to manage their configuration and data.

As mentioned by another user, there should be an option to allow unpacking a WAR located outside of the Webapps.

It was mentioned in another thread the reason it isn't so is it complicate the autodeploy.  Well, make those feature mutually exclusive. You cannot use autoDeploy if unpackWarOutsideWebapp is true.  I know we don't use autoDeploy for security reason.

We're strongly thinking of stopping the Tomcat 7 migration project and stick to Tomcat 6 until the EOL if this 'fix' is not rollabacked or offer a workaround.

Nowhere in the documentation of Tomcat 6 was it mentioned the war need to be in the Webapp to be unpacked. I haven't found any specification mentioning that this has to be that way too. So changing this behaviour without first consulting people and on an haunch really was a bad move.

If it's working, don't fix it.  Right now, it's broken for a tomcat instance fully relying on configuration files to manage context and deployment.
Comment 10 Mark Thomas 2012-02-02 16:24:46 UTC
(In reply to comment #9)
> Nowhere in the documentation of Tomcat 6 was it mentioned the war need to be in
> the Webapp to be unpacked. I haven't found any specification mentioning that
> this has to be that way too. So changing this behaviour without first
> consulting people and on an haunch really was a bad move.

The Tomcat documentation for the host element has contained variations of the following since the 6.0.0 tag (and probably earlier but I didn't go back into the 5.5.x series):
<quote>
upackWARs: Set to true if you want web applications that are placed in the appBase directory as web application archive (WAR) files to be unpacked into a corresponding disk directory structure, false to run such web applications directly from a WAR file.
</quote>

The Servlet specification is quite clear that applications should not rely on running in expanded form on a file system. If I recall correctly, it mentions that they might be located in a database.

The behaviour if the WAR is placed outside the appBase was undefined until 7.0.12 where it was defined that they would not be expanded due, as you say, to complications this creates for auto-deployment.

The simplest work-around is to deploy as an expanded directory rather than as a WAR or deploy into the appBase.


I'm moving this issue to an enhancement. Any patch that addresses this needs to take account of the various edge cases as per the referenced discussions.
Comment 11 aash 2012-05-24 18:03:55 UTC
This is a huge deal.  It totally screws with our ability to manage our war versions.  Why go fixing something that isn't broken?   You just removed our ability to do hot fixes and tests on a live system, and still be able to manage our war versions.

Can anyone demonstrate how to unpack an external war to the ROOT context?  I was trying that as an alternative, and nothing worked.  It worked fine unpacking to any other context, but that's utterly useless for me.
Comment 12 Pid 2012-05-24 20:39:01 UTC
Have you tried using the 'jar' command?
Comment 13 aash 2012-05-24 20:47:41 UTC
(In reply to comment #12)
> Have you tried using the 'jar' command?

yes.. when i unpack into a  webapps/foobar it works fine.  the problem is that is must run on the root context, so when i unpack it into webapps/ROOT, it ignores it.
Comment 14 Pid 2012-05-24 21:09:05 UTC
That sounds like a different problem, please ask on the Tomcat Users list.
Comment 15 jwa 2012-09-06 15:26:27 UTC
Not expanding and having no option to expand external war files is a major inconvenience for us as well.
The fact that the unpackWAR attribute in the context, when explicitly set to true, does not result in the war being unpacked is very unintuitive.  It would normally expect that an explicit setting on that should trump all other defaults/behavior.

However as a workaround, I have found that making an appropriately named symlink inside the webapps directory pointing to the external war seems to suffice.  This may be an acceptable long term solution.
Comment 16 Andreas Horstmann 2013-04-18 22:02:44 UTC
If using a relative path in docBase attribute, unpackWARs will still work in Tomcat 7. Should also be working on absolute paths.
Comment 17 Mark Thomas 2013-04-18 22:26:07 UTC
(In reply to comment #16)
> If using a relative path in docBase attribute, unpackWARs will still work in
> Tomcat 7. Should also be working on absolute paths.

No, it should not work with relative paths (until this enhancement is implemented) for the same reasons it was disabled for absolute paths. See the thread in comment #8 for the explanation of why this was disabled.
Comment 18 Andreas Horstmann 2013-04-19 09:24:31 UTC
But it does work. I used it at one of my customers and I also reproduced the behavior locally.
Comment 19 Charlie Hubbard 2013-05-12 23:05:09 UTC
The problem is deploying to ROOT context has only a handful of options and these options are difficult to use.  And this is the most common context to deploy to.  What we want is the ability to copy our war file up to a directory without manually manipulating it (ie renaming it, unpacking it, etc), copy a file to a path to map it to a context without having to modify a file owned by TC, and let tomcat unpack it for us.

The only options we have right now are:

   1. Rename our war file to ROOT.war and put it in the webapps folder.
   2. Put the war file external to webapps, and put an application context descriptor file under Catalina/localhost to point to the external war mapping it to the root context.  
   3. The final option is to modify server.xml file to add the context mapping and put the external war file in there.

Well the first option sucks and really the only one that works now because we can't keep our name and version information intact.  No build process out there outputs ROOT.war so that's a manual step we have to perform to make this work.  And you have do several steps on the server to make it work.  You can't just use scp to copy the file up and restart.

The second option sucks because TC will let you set the ROOT context using a file under Catalina/localhost, but it refuses to extract the war file because its an external war file.

The third option sucks because you can't automate easily, but also suffers from external war file unpacking problem.  And this is in bold to not do in the TC docs.

What is left writing lots of specialized deployment code to force us to manually extract the war files ourselves.  Why not just make the whole process straight forward?  What are all of these features not uniform and straight forward?  I don't think the problem is external vs internal unpacking.  It's the whole freaking process of deploying a war file and mapping that onto a URL of our choosing without modifying the internals of TC or changing our output.

What would be super straight forward and simple is.  Copy the war file up to a folder (maybe webapps, any folder, etc), and deploy a config file (like a context file) that can map the war file to a URL including ROOT path, specify if a war file should be unpacked or not, redeploy, etc.  Something like this:

<Context url=""
         docBase="/opt/somepath/myspecialapp-1.0.war"
         unpack="true"
         redeploy="true" 
         debug="1" 
         privileged="true"/>

Or relative to webapps

<Context url=""
         docBase="myspecialapp-1.0.war"
         unpack="true"
         debug="1"
         priviledged="true"/>

It just so straight forward it makes me want to cry that TC right now is so confusing with all of these terrible options we have.  If you make the process straight forward the code will clean itself up because it's simple.  I think TC has so many options for deployment that are worthless it made the code complex.  It's just you took away one of the useful features for bad reasons.
Comment 20 Charlie Hubbard 2013-05-12 23:26:34 UTC
Why didn't you just clean up the docs and make the docs match the code instead of changing the code to match the docs?  Seems like a lot less work to type in a paragraph in the docs than change code!  Look at how much more typing everyone on the internet has now done in response to this change!  The rest of the reasons for changing code that I read didn't make a compelling reason to change it other than it might have simplified the code.  Well of course it did a feature was removed.  It's not like it was refactored.  Code was just deleted.
Comment 21 Mark Thomas 2013-05-13 08:07:18 UTC
(In reply to comment #19)
If the requirement is to just copy the WAR, deploy it as ROOT and retain version information in the file name you can name the file ROOT##version-information-here.war

The version marker after the ## is used in parallel deployment but it can still be used even if you have a single version deployed and specifying it in that case has no negative consequences.

> Why not just make the whole process straight forward?

I'd love to but you aren't the only user of Tomcat. What might seem simple to you may be the exact opposite of what another user wants.

> What are all of these features not uniform and straight forward?

I do think there are issues with clarity and consistency with automatic deployment. This stems from a lack of definition of how automatic deployment is meant to behave (particularly around the many edge cases) and also from the changes over time that have tried to address the issues for one group of users that have usually created problems for another.

> I don't think the problem is external vs internal unpacking.

Then you haven't fully understood the problem. There are additional edge cases created by supported unpacking of external WARs that need to be thought through. 

> What would be super straight forward and simple is.  Copy the war file up to
> a folder (maybe webapps, any folder, etc), and deploy a config file (like a
> context file) that can map the war file to a URL including ROOT path,
> specify if a war file should be unpacked or not, redeploy, etc.

Breaking the linkage from XML / WAR / DIR name to context path creates multiple problems for automatic deployment around name clashes that would require far more complexity to solve. I don't see any implementation along those lines ever being accepted.

> It just so straight forward it makes me want to cry that TC right now is so
> confusing with all of these terrible options we have.  If you make the
> process straight forward the code will clean itself up because it's simple. 

I agree a simple process will result in simple code but what you are proposing is not simple.

> I think TC has so many options for deployment that are worthless it made the
> code complex.

Again, you are not the only user of Tomcat. There are other users that do find the features useful and they presented a strong enough case to the community to convince the committers that the feature would be of useful.

> It's just you took away one of the useful features for bad reasons.

I removed an undocumented feature you had been relying on because of the complications is caused for automatic deployment. You might not like that decision but the tone of your comment is not likely to help your case.

I'd like to remind folks reading this that this is open source with an open development process. If you want to see this enhancement request implemented, constructive contributions are far more likely to achieve the results you desire. A discussion [1] is already underway on the dev list about proposed changes [2] to automatic deployment covering:
- creating a clear definition of the expected behaviour of automatic deployment
- ensuring that the behaviour of automatic deployment is logical and consistent
- implementing this enhancement request

If you want to move this enhancement request forward, constructive contributions to that thread and constructive criticisms of the proposed new behavior are the way to do it.

[1] http://tomcat.markmail.org/thread/zmcxxfqbb3p4qft5
[2] http://people.apache.org/~markt/dev/auto-deployment-proposed.txt
Comment 22 Mark Thomas 2013-05-13 08:32:56 UTC
(In reply to comment #20)
> Why didn't you just clean up the docs and make the docs match the code
> instead of changing the code to match the docs?

Again, because this undocumented behavior created a lot of edge cases that weren't handled by the automatic deployment code.

> Seems like a lot less work to type in a paragraph in the docs than change code!

The options, with the one requiring the least effort first were:
1. add a check to the code to prevent the undocumented behaviour
2. modify the docs to reflect the current behaviour highlighting the various edge cases that it could cause and explaining how to avoid them
3. implement the requested feature properly with the edge cases all thought through and update the docs to add the new feature

1 was implemented at the time. This enhancement request is for 3.


> Look at how much more
> typing everyone on the internet has now done in response to this change! 

And it is a pity so little of it has been constructive. If the same efforts had been directed towards constructive ends this issue would be closer to being resolved.

> The rest of the reasons for changing code that I read didn't make a
> compelling reason to change it other than it might have simplified the code.

You misunderstand the simplification argument. Prior to the change, external WARs were unpacked but this was neither the intended nor the documented behaviour. Unpacking external WARs creates additional edge cases that needed to be thought through and correctly handled. That would have required a reasonable amount of change to code that is already fairly complex. The simpler option was to prevent this unintended behaviour until such time as this feature could be properly implemented.

> Well of course it did a feature was removed.  It's not like it was
> refactored.  Code was just deleted.

The above statement is incorrect. Clearly you made an assumption rather than looked at the source code to see how the change was implemented [1]. Making assumptions like that only serve to damage your credibility and undermine any valid arguments you may be making.

[1] http://svn.apache.org/viewvc/tomcat/tc7.0.x/tags/TOMCAT_7_0_12/java/org/apache/catalina/startup/ContextConfig.java?r1=1079769&r2=1080719&diff_format=h
Comment 23 Mark Thomas 2013-05-14 10:32:52 UTC
An initial implementation of these feature has been added to 8.0.x. It is expected that the implementation will evolve as the test cases are expanded to cover the various edge cases. Once the implementation is complete it will be considered for back port to 7.0.x.
Comment 24 Mark Thomas 2013-05-17 19:09:54 UTC
The implementation work is complete in trunk. Now is the time to test it and provide feedback. It will be proposed to back-ported to 7.0.x after a suitable period (at least several weeks - probably longer) has passed to give folks a chance to thoroughly review the changes.
Comment 25 Konstantin Kolinko 2013-11-25 22:51:09 UTC
These fixes are currently being ported to Tomcat 7, to be in 7.0.48.

A note: Do not forget to update the note in the migration guide, once this fix is completed.
http://tomcat.apache.org/migration-7.html#Deployment
Comment 26 Mark Thomas 2013-11-26 00:39:21 UTC
The back-port of the automatic deployment refactoring has been completed. Support for this feature was part of that refactoring. It will be included in 7.0.48 onwards.
Comment 27 Andrei Zhuk 2014-08-12 09:48:11 UTC
The wars outside appBase are not unpacked in 7.0.55 although worked in 7.0.54. Can you please clarify if it is an intentional change or a regression?
Comment 28 Konstantin Kolinko 2014-08-12 10:11:34 UTC
(In reply to Andrei Zhuk from comment #27)
> The wars outside appBase are not unpacked in 7.0.55 although worked in
> 7.0.54.

There is no such change. I have just tested it with 7.0.55 and the feature works. 

(I tested both with a separate context xml file and with Context element in server.xml. Both deployments successfully unpacked a war.)

Ask on the users mailing list if you still have problems.
Comment 29 Andrei Zhuk 2014-08-12 10:35:30 UTC
My bad - there was a problem with appBase directory permissions. Everything works. Sorry for confusion.