Bug 10755 - Ant 1.5's jar task fails to add new files when run multiple times
Summary: Ant 1.5's jar task fails to add new files when run multiple times
Status: RESOLVED FIXED
Alias: None
Product: Ant
Classification: Unclassified
Component: Core tasks (show other bugs)
Version: 1.5
Hardware: PC other
: P3 critical with 22 votes (vote)
Target Milestone: 1.5.2
Assignee: Ant Notifications List
URL:
Keywords:
: 10823 11039 11209 11623 11699 11756 12028 13783 14044 14563 14586 15304 15906 17684 21082 (view as bug list)
Depends on:
Blocks:
 
Reported: 2002-07-12 19:38 UTC by Sven-J
Modified: 2007-04-02 06:06 UTC (History)
23 users (show)



Attachments
Suggestion on how to fix this bug. (1.21 KB, patch)
2002-10-23 09:17 UTC, peter.holzwarth
Details | Diff
patch for the task Zip (9.05 KB, patch)
2003-01-04 06:48 UTC, Antoine Levy-Lambert
Details | Diff
patch for the ZipScanner file (2.99 KB, patch)
2003-01-04 06:49 UTC, Antoine Levy-Lambert
Details | Diff
Patch for the Jar task (1.82 KB, patch)
2003-01-04 06:50 UTC, Antoine Levy-Lambert
Details | Diff
Test setup (2.37 KB, application/octet-stream)
2003-01-24 11:00 UTC, peter.holzwarth
Details
Ok, found the interesting test case now... See next comment. (2.99 KB, application/octet-stream)
2003-01-24 14:29 UTC, peter.holzwarth
Details
The attached zip contains the log output from my WinXP installation. (11.33 KB, application/octet-stream)
2003-01-24 16:29 UTC, peter.holzwarth
Details
ned (deleted)
2007-04-02 06:06 UTC, Nekto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sven-J 2002-07-12 19:38:05 UTC
Ant 1.5's <jar/> task doesn't add new files to a Jar archive if it's run 
multiple times on the same file in the same Ant process. It seems that there is 
something wrong with the dependency checking of the archive contents. This 
behaviour was first seen with Ant 1.5, after using 1.5Beta3 and 1.4.1 without 
problems.

To reproduce:
1. Copy the enclosed build script sample into a file.
2. Run Ant 1.5 on it:
   $ ant -v 
3. Check the archive contents, test.jar:
   $ unzip -l test.jar 
4. The archive is still empty (except for the generated manifest file), but
   a single file should have been added by the second <jar/> invocation.

<?xml version="1.0"?>
<project name="Expose a Bug in Ant's Jar Task" default="jar" basedir=".">

  <target name="jar">
    <jar jarfile="test.jar"
      basedir="."
      excludes="*"/>

    <jar jarfile="test.jar"
      basedir="."
      includes="build.xml"
      excludes="test.jar"/>

  </target>

</project>
Comment 1 Klaus Ruehl 2002-07-18 14:05:44 UTC
*** Bug 10823 has been marked as a duplicate of this bug. ***
Comment 2 Matt McHenry 2002-07-18 16:11:40 UTC
I'm seeing basically this same problem with the <zip> task, which is the
superclass of the <jar> task.

I backed out the changes of rev. 1.78.2.6 from
org/apache/tools/ant/taskdefs/Zip.java and things started working again. 
According to the CVS logs, these changes were made to address bug 8069.
Comment 3 Stefan Bodewig 2002-07-24 11:59:09 UTC
Yes.

In Ant 1.4.1 <zip>'s update didn't work - it would always update the zip file.

Now in 1.5 it doesn't work as expected, it will only update the zip file if the
files you want to add are newer than the archive.  This is consistent with
<zip> not recreating an archive if the archive is newer than the files you'd
like to see in it when update is false.

Comment 4 Stefan Bodewig 2002-07-24 12:03:27 UTC
*** Bug 11039 has been marked as a duplicate of this bug. ***
Comment 5 Holger von Thomsen 2002-07-24 12:35:11 UTC
Hi!

This is not exactly the same as described in Bug #11093, which has been marked 
as a DUPLICATE this one.

After evaluating the exact issue behind Bug #11093, I found out, that updating 
a WAR with a file, which already exists in the WAR, works fine on Windows NT/2K 
and SuSE-Linux 7.3 Intel, Kernel 2.4.x, but fails on SuSE-Linux 7.0 zSeries 
IBM, Kernel 2.2.16. I think this could be related to the different versions of 
JDKs. Windows and Intel-Linux are using Sun JDKs, but the zSeries needs an IBM 
JDK. This probably should become a new Bug.
Comment 6 Peter Hale 2002-07-29 15:10:24 UTC
I have a simalr situation when trying to add files to a jar using <jar> and 
<zip>. The first task adds files properly bu the second produces an error for 
each file to be added, stating that the target jar is "up to date" and the file 
to be added is "skipped".

Target looks like this:
    <target name="jar" depends="compile" description="Create client jar file">
        <jar destfile="${jar.file.name}.jar" includes="**" 
basedir="${dest.dir}" manifest="ant/dashclient-nosign.mf" update="true" />
        <zip destfile="${jar.file.name}.jar" update="true" >
            <zipfileset src="shared/Cnnxn/CnnxnUtils.jar" excludes="META-
INF/MANIFEST.MF" />
        </zip>
    </target>


I was able to workaround the problem by adding a <touch> task between the <jar> 
and <zip> tasks using a date that I know is in the past.

    <target name="jar" depends="compile" description="Create client jar file">
        <jar destfile="${jar.file.name}.jar" includes="**" 
basedir="${dest.dir}" manifest="ant/dashclient-nosign.mf" update="true" />
        <touch file="${jar.file.name}.jar" datetime="06/28/2000 2:02 pm"/>
        <zip destfile="${jar.file.name}.jar" update="true" >
            <zipfileset src="shared/Cnnxn/CnnxnUtils.jar" excludes="META-
INF/MANIFEST.MF" />
        </zip>
    </target>

Peter Hale phale@cnnxn.com
Comment 7 Stefan Bodewig 2002-07-30 14:15:01 UTC
*** Bug 11209 has been marked as a duplicate of this bug. ***
Comment 8 Tony Ng 2002-08-12 05:40:16 UTC
*** Bug 11623 has been marked as a duplicate of this bug. ***
Comment 9 Anders Cederlund 2002-08-13 12:35:39 UTC
We have a similar problem using the zip task.
We have some files that should be compressed in the
zip archive and some that are just stored. (.gifs)

When ant stores the files it reports the archive as
up to date and nothing was stored.

If I use the touch workaround, it stores the the files
allright BUT it uncompress all existing files and
stores them uncompressed.

(Btw, touch millis="10" does not work as a workaround.)
Comment 10 Stefan Bodewig 2002-09-03 16:00:32 UTC
*** Bug 11699 has been marked as a duplicate of this bug. ***
Comment 11 Stefan Bodewig 2002-09-03 16:02:17 UTC
*** Bug 11756 has been marked as a duplicate of this bug. ***
Comment 12 Stefan Bodewig 2002-09-03 16:05:38 UTC
*** Bug 12028 has been marked as a duplicate of this bug. ***
Comment 13 Stefan Bodewig 2002-10-21 15:17:25 UTC
*** Bug 13783 has been marked as a duplicate of this bug. ***
Comment 14 peter.holzwarth 2002-10-23 09:17:38 UTC
Created attachment 3574 [details]
Suggestion on how to fix this bug.
Comment 15 peter.holzwarth 2002-10-23 09:20:08 UTC
To address the issue that a .zip or .jar cannot be updated depending on an
existing or outdated entry in the archive, I've started digging into the Mergers
and Scanners that decide about this magic, and have added a few lines to
SourceFileScanner.java that make it actually look into the archive, which as far
as I can see noone did before. Since the documentation says that "New files are
added; old files are replaced with the new versions.", I believe that this is
the correct (and most useful) behaviour. The code may not be the most beautiful
piece in the world, but it solves the problem perfectly, at least for zips and
jars. I have not tested it for tars. I hope that the attached cvs diff can be
used to fix this issue generally.

-- best regards,
Peter Holzwarth
mailto:peter.holzwarth@io-software.com
Comment 16 Stefan Bodewig 2002-10-29 11:20:43 UTC
*** Bug 14044 has been marked as a duplicate of this bug. ***
Comment 17 Stefan Bodewig 2002-11-15 11:10:45 UTC
*** Bug 14563 has been marked as a duplicate of this bug. ***
Comment 18 Stefan Bodewig 2002-11-15 12:32:11 UTC
*** Bug 14586 has been marked as a duplicate of this bug. ***
Comment 19 Stefan Bodewig 2002-11-19 13:06:58 UTC
Peter, thank you for the patch - unfortunately it won't work that easy.

Apart from the fact that there are more tasks that use MergeMappers, your patch
wouldn't take things like the prefix and fullpath attributes of <zipfileset> into
account.

I agree that SourceFileScanner is the way to go and I'm going to look into it
in the next few days, but I'm going to approach it from a different angle.  I'll
probably add a new attribute to zip that needs to be set to actually look into
the archive - one could argue that looking into the archive should be the default
for update="true".
Comment 20 Stefan Bodewig 2002-12-12 11:02:10 UTC
*** Bug 15304 has been marked as a duplicate of this bug. ***
Comment 21 Antoine Levy-Lambert 2003-01-04 06:48:01 UTC
Created attachment 4334 [details]
patch for the task Zip
Comment 22 Antoine Levy-Lambert 2003-01-04 06:49:49 UTC
Created attachment 4335 [details]
patch for the ZipScanner file
Comment 23 Antoine Levy-Lambert 2003-01-04 06:50:47 UTC
Created attachment 4336 [details]
Patch for the Jar task
Comment 24 Antoine Levy-Lambert 2003-01-04 06:59:56 UTC
I have done a smoke test concerning these 3 patches.
When one does what Sven Karlsen describes 
(build first an empty jar, then add one file for instance build.xml), the jar tasks notices that 
build.xml is missing the second time and rebuilds it.
I have also done a smoke test with a zipfile 
task containing zipfilesets, similar to what is in the ant manual, including a zipfileset coming 
from a zip file.
Here is what I have played with :
<?xml version="1.0"?>
<project 
name="Expose a Bug in Ant's Jar Task" default="jar" basedir=".">
<!-- <touch file="build.xml" 
datetime="06/28/2000 2:02 pm"/> -->
  <target name="jar">
    <jar jarfile="test.jar"
      
basedir="."
      excludes="*"/>

    <jar jarfile="test.jar"
      basedir="."
      
includes="build.xml"
      excludes="test.jar"/>

  </target>
  <target name="jar2">
    
<jar jarfile="test.jar"
      basedir="."
      includes="build.xml"
      
excludes="test.jar"/>

    <jar jarfile="test.jar"
      basedir="."
      
includes="build.xml"
      excludes="test.jar"/>

  </target>
  <target name="zip">
    
<touch file="htdocs/manual/inputhandler.html" datetime="06/28/2000 2:02 pm"/>
      <zip 
destfile="manual.zip">
         <zipfileset dir="htdocs/manual" prefix="docs/user-guide"/>
         
<zipfileset dir="." includes="ChangeLog27.txt" fullpath="docs/ChangeLog.txt"/>
        
<zipfileset src="examples.zip" includes="**/*.html" prefix="docs/examples"/>
    
</zip>
    <!-- <echo message="Hello, world" file="htdocs/manual/text.html"/> -->
      <zip 
destfile="manual.zip">
         <zipfileset dir="htdocs/manual" prefix="docs/user-guide"/>
         
<zipfileset dir="." includes="ChangeLog27.txt" fullpath="docs/ChangeLog.txt"/>
        
<zipfileset src="examples.zip" includes="**/*.html" prefix="docs/examples"/>
    
</zip>
    <delete file="htdocs/manual/text.html" quiet="true"/>
  </target>
</project>
Comment 25 Stefan Bodewig 2003-01-09 08:06:18 UTC
*** Bug 15906 has been marked as a duplicate of this bug. ***
Comment 26 Stefan Bodewig 2003-01-09 08:09:50 UTC
A quick update to let you know that this bug has not been forgotten:

http://marc.theaimsgroup.com/?l=ant-dev&m=104209947417287&w=2

working on it ...
Comment 27 Thomas Singer 2003-01-21 12:36:09 UTC
For me the problem is, that I want to use the "update" option to add more files to the jar, but because 
these files are older than the freshly created jar, they won't be added. This assumptation is 
wrong. These test should only happen, if the files to add are already in the jar; if they aren't, 
they must be added - no matter, what file-time they have.

Alternative: provide another option 
to *add* files.
Comment 28 Stefan Bodewig 2003-01-23 15:38:32 UTC
For some preliminary feedback, could some of you please try the nightly build of
<http://cvs.apache.org/builds/jakarta-ant/nightly/> 2003-01-22 or later?

The <jar> task is now supposed to update the archives correctly.
Comment 29 peter.holzwarth 2003-01-23 17:06:21 UTC
Should this now also do an update if a file in the archive is older than a 
source file, but the archive is newer? A first test didn't work, so I'd like to 
ask before we spend more time.
Comment 30 Stefan Bodewig 2003-01-23 17:13:16 UTC
Yes, it should add files that are older than the archive as well as update
the archive if the files inside are older than the files to add/update.

If it doesn't work, I have some more work to do 8-(

A build-file snippet might help.  As I said, I wanted some preliminary feedback,
don't spend too much time with it.

Thanks.
Comment 31 Antoine Levy-Lambert 2003-01-23 17:16:23 UTC
To my opinion, in the current CVS head version, the date time of the archive itself is irrelevant. 
What is being compared are the date/time of the archive entries versus the date/time of the files 
to be archived.
Comment 32 John Lawrence 2003-01-24 00:01:11 UTC
I have a long classpath (990 characters).  I wanted to test the fix, but I 
haven't been able to.

When ant_home points to my 1.5.1 ant directory, and I run "ant -version", I get

Apache Ant version 1.5.1 compiled on October 2 2002

If I change ant_home to point to 1.6alpha, I get

The input line is too long.
:gotAllArgs
 was unexpected at this time.

Changing the path so that the 1.6alpha ant.bat is run doesn't change the result.

 

 
Comment 33 Stefan Bodewig 2003-01-24 10:19:01 UTC
For a test, you could remove all the ant-*.jar files from the lib directory as
optional.jar should contain all of their contents anyway.
Comment 34 peter.holzwarth 2003-01-24 11:00:59 UTC
Created attachment 4530 [details]
Test setup
Comment 35 peter.holzwarth 2003-01-24 14:29:10 UTC
Created attachment 4537 [details]
Ok, found the interesting test case now... See next comment.
Comment 36 peter.holzwarth 2003-01-24 14:32:20 UTC
Testing of files in folders seems not to happen. Could be caused by a missing 
.replace('\\', '/') for Windows folders ? I haven't looked into the source so 
far, so this is just a suggestion. See the tests in Attachment id=4537. They 
show that the behaviour is not what we would expect.
Comment 37 Stefan Bodewig 2003-01-24 14:46:13 UTC
I had already found some problems with directory entries themselves - and fixed
them.

Thank you for the test cases, I'll look into them.
Comment 38 Stefan Bodewig 2003-01-24 15:02:49 UTC
Peter, what are the result you are getting?  Can you give me the -debug output
from using the 1.6alpha code?

I'm not using Windows myself and it seems to work for me.  This could be a platform
issue or mean that my latest patches have fixed it.
Comment 39 peter.holzwarth 2003-01-24 16:29:03 UTC
Created attachment 4538 [details]
The attached zip contains the log output from my WinXP installation.
Comment 40 Stefan Bodewig 2003-01-27 13:32:06 UTC
Peter, we have a combination of problems here.

First, there has been a platform issue, that should be fixed now.  Could you 
please retry with the next nightly build (-01-28) or use this one
<http://cvs.apache.org/~bodewig/ant.jar>?

The second problem is your test with a file that is only half a second newer than
the one stored last time around, this is not going to work by any means.

ZIP archives store time with a granularity of two seconds.  If we want to avoid
an "always updates" problem, we need to round up to the next higher even number
of seconds when storing the entries - this may lead to situations where <zip> and
friends will not update an archive for a newer file, if that file is less than
two seconds newer than the originally stored one.  I don't see a solution here.

Comment 41 peter.holzwarth 2003-01-29 08:38:26 UTC
The 01-28 build works fine for me.

If I could vote for a behaviour in situations where the file to be updated and 
the file in the archive are in the same 2 seconds interval, I would prefer 
doing the update always in this case since this seems to be the best guess one 
can make.
Comment 42 Scott Ellsworth 2003-01-29 21:36:10 UTC
Were we to always add a file if its date stamp were within two seconds, we would always 
be adding files even if they duplicated what is in the archive, which is often not what is 
desired.

We can get this behavior, I believe, from the update attribute.  If it is set to "false', will it not 
always add the file?

(If I am wrong, then I can see the need for a "force" attribute that always replaces.)
Comment 43 Stefan Bodewig 2003-02-19 13:55:20 UTC
Finally - fixed.

Please use the latest build from http://cvs.apache.org/~bodewig/gump/ant1.5.2alpha/
to confirm that zip and friends now work as expected.  What you find there is
quite close to what will be Ant 1.5.2beta1, but doesn't include all optional tasks.
Comment 44 peter.holzwarth 2003-02-21 13:51:40 UTC
The tasks in 1.5.2alpha work great for us! Thanks!

-- Peter Holzwarth.
Comment 45 Stefan Bodewig 2003-03-07 07:45:40 UTC
*** Bug 17684 has been marked as a duplicate of this bug. ***
Comment 46 Ronak Shah 2003-03-07 14:28:29 UTC
I was using the nightly build from Jan 28th until just recently, and that 
version worked as I expected the <jar update="true"> task to work:  It added 
files regardless of date and never removed files already existing in the jar.

I've just tried it with the 1.5.2 build, and in all my projects save one, it 
works fine.  But in one case I am finding that it is removing some pre-existing 
files (my ejb-jar.xml, which is how I noticed)

If necessary I can provide more test information.
Comment 47 Stefan Bodewig 2003-03-07 14:38:54 UTC
Well, I introduced a new most-popular bug in 1.5.2's <zip> 8-(

How do you add your ejb-jar.xml?  I bet it uses <zipfileset>s internally in some
way.  My guess is that the file is not really missing, you just don't see it
in WinZIP (could you try "jar tf filename" instead?)

If I'm correct, your report is a duplicate of bug 17648 and - thank god - no
regression of this one here.
Comment 48 Ronak Shah 2003-03-07 15:13:55 UTC
You are correct.  I'll set this back to fixed and verify 17648 next week.  
Thanks.
Comment 49 Stefan Bodewig 2003-06-25 13:45:07 UTC
*** Bug 21082 has been marked as a duplicate of this bug. ***
Comment 50 Christian Schubert 2004-10-21 15:06:39 UTC
*** Bug 13783 has been marked as a duplicate of this bug. ***
Comment 51 Nekto 2007-04-02 06:06:18 UTC
Created attachment 19869 [details]
ned