ant seems to produce jar files which are exactly zip files. However, Sun's jar utility seems to produce jar files with a special 0xcafe tag. The Solaris kernel uses this tag to help determine if a jar file should be executable. For example, a jar created using jar(1) can be run: $ chmod a+x hello.jar $ ./hello.jar Hello, World! Whereas the same jar, produced by ant, cannot be. The difference seems to come just after the initial MANIFEST declaration: /usr/bin/jar: 0000000: 504b 0304 1400 0800 0800 7c23 8b31 0000 PK........|#.1.. 0000010: 0000 0000 0000 0000 0000 0900 0400 4d45 ..............ME 0000020: 5441 2d49 4e46 2ffe ca00 0003 0050 4b07 TA-INF/......PK. ^^^^^ 0xcafe ant: 0000000: 504b 0304 0a00 0000 0000 0c25 8b31 0000 PK.........%.1.. 0000010: 0000 0000 0000 0000 0000 0900 0000 4d45 ..............ME 0000020: 5441 2d49 4e46 2f50 4b03 0414 0008 0008 TA-INF/PK....... ^^^^^ 'PK' A frustrating part of this is that Sun's JAR file page doesn't mention this behavior. Since I work for Sun, I'll file a bug on that, and see if I can get the java team to better document this behavior. I also found this page, which might help (see putNextEntry()): http://www.javaresearch.org:88/source/jdk142/java/util/jar/JarOutputStream.java.html
Just a quick note - please do not post links to confidential files into the Apache bugzilla. The copyright at the top of this file does not make it suitable for us to look at. Sorry. I had to quickly turn away before I turned to a pillar of salt.
Conor's comment scared me so much that I didn't dare to follow your link 8-) From the local file header snippets you send, it seems as if Solaris expects the entry for META-INF/ to contain an extra field with a header ID of 0xcafe and no additional content. Can you get somebody to confirm this without breaching any NDAs? Other differences I see is jar uses DEFLATED for directories, Ant uses STORE - could you ask your secret sources whether this is significant?
Stefan, well there's no real reason to be concerned, it's just the class, JarOutputStream, from src.zip, I think. It does say SUN CONFIDENTIAL, but I guess you can look at it.
Unless I ever wanted to contribute to CLASSPATH or Kaffe, that is.
Dan, If your OS is relying on a non documented feature of JAR files, something that is only an artifact of the implementation, not the spec, then, it aint going to work with Ant built products. If it is some part of the formal JAR spec, then we are deficient in not supporting it, and no doubt we should consider a fix. But we need a formal, proper, statement in the JAR spec, so we can point to it and say "this was the correct action". We field a lot of grief about manifest handling (esp on the classpath) attribute, and need that specification to point to.
Fair enough. I'll see if I can find someone on the Java team to provide a clarification. We have a pretty careful interface management process within the Solaris process; so if we're interacting with jar(1) in an undocumented way, I guess we need to figure that out. Please give me a week or so to do some further investigation.
Hi-- as an update. The responsible parties (some from Java, some from Solaris) are talking about this issue, and reviewing the case materials which document the interface contract between Java and Solaris. It appears that this won't get resolved completely until one of the parties returns from vacation, on 1/10/2005. So my estimate of a week was too optimistic. So: we'll have an update sometime in January.
We can handle a delay, after all, we have no schedule for the next release. One question: are you trying to look for a magic number at an offset in the binary, or are you parsing the zip file? A magic number pattern is most like classic unix exe lookup, but is inordinately brittle against different legitimate layouts of ZIP files. You should really be recognising the PK DWORD at the beginning of the file and say 'this is a zip, lets look inside it *using the zip algorithm* and see if it has a manifest. Presence of a valid manifest is what should really determine if a jar is executable. If that aint there, you wont know which class to execute, etc, etc. Since we do produce valid manifests if asked to, that should all work.
(In reply to comment #8) > We can handle a delay, after all, we have no schedule for the next release. > One question: are you trying to look for a magic number at an offset in the > binary, or are you parsing the zip file? > > A magic number pattern is most like classic unix exe lookup, but is inordinately > brittle against different legitimate layouts of ZIP files. You should really be > recognising the PK DWORD at the beginning of the file and say 'this is a zip, > lets look inside it *using the zip algorithm* and see if it has a manifest. > > Presence of a valid manifest is what should really determine if a jar is > executable. If that aint there, you wont know which class to execute, etc, etc. > Since we do produce valid manifests if asked to, that should all work. Thanks for your patience. The algorithm as it stands is as follows: The kernel decides to use the 'javaexec' file executor based on magic number (same as with ELF and interpreted files, like #!... ). Based on the magic number (PK\003\004), the javaexec module is invoked. My reading of the code is as follows: 1. Read the 30 byte file header. 2. Look for 'PK\003\004' at the beginning. If not, fail. 3. Let XOFF = 30 (file header size) + Size of the first file's name (stored at offset 26). So, if the first file is META-INF/, XOFF = 9. 4. Let XOFF_END = XOFF + Size of the first file's extension. Normally this would be zero, but in this case, the jar tool seems to set it in the archive to '4'. 5. Scan the buffer starting at XOFF and ending at XOFF_END for the byte pattern: 0xcafe. If not found, fail. 6. Find and invoke the Java runtime environment on the file. I can only assume that the answer to: "why not scan the whole file for the manifest file" is that this would be rather expensive, although I'm not sure. If you mean that we should also unzip the manifest file and parse it-- I think the answer is that we try not to sink that level of complexity into the kernel.
I agree that the OS doesnt need to get into parsing a zip file, but worry if it doesn't, your solution will be too brittle. Are there any other zip filetypes which are executable? Why not hand off to a launcher program that launches any PK exe; for java it would look for the manifest and run from there.
Dan, from your reading of the code it should be the very first entry of the ZIP that gets the magic extra field, not necessary META-INF/. We'll be happy to adjust Ant if you can get some kind of official statement. Thanks.
Any news?
I've found a document that mentions the magic number. See the last paragraph in <http://java.sun.com/j2se/1.4.2/docs/guide/jar/jarGuide.html>. Now we only need to find a public file that says what the value of magic number is and where it has to go. GNU file 4.13 doesn't recognize it. Hmm, Solaris 10 sources?
I decided to no longer search around or wait for official statements. The CVS HEAD version of Ant now adds the CAFE marker to the very first entry of a jar file (which will always be the META-INF/ directory in Ant's case). Could you please try a nightly build of 2005-03-12 or later and close this bug report as fixed if it works?
Assuming fixed