Steve,
does this problem also arise when you run the java app from the command line? 
It may be a java 1.4 issue, cos when ant runs stuff ina  new jvm, it just 
execs java

1. running the demo app from the command line works fine 
2. running the demo app as a sub-process created by a parent 
   process where the parent is executed from the command line
   works fine
3. running the demo app from the Ant target <java ... fork="false"> 
   works fine
4. running the demo app from the Ant target <java ... fork="true"> 
   fails with the subprocess still in existance after the parent
   process has terminated

Given that process destruction happens when running from the command line and 
does not happen when running with the <java ... fork="true"/> suggests that 
this is an Ant bug and not a JVM bug (even through it feels like a JVM bug) - 
bottom line is that I have not been able to create the same fail condition 
outside of Ant.

Have updated the demo on http://home.osm.net/020423.html to demonstrate this.
Once built the non-fail command line demo can be run using:

   $ java -classpath test.jar Main
   
With the following output:

   main
   MAIN: sub-process created: java.lang.Win32Process@f4a24a
   MAIN: shutdown hook set

The hit ^C

   MAIN: shutdown hook fired
   MAIN: subprocess destroyed

Invoking <ant fork="true" ... > should result in similar behaviour.  I.e. the 
forked java sub-process should terminate - instead it continues until manually 
destroyed using the task manager.

Could you please try a recent nightly build of Ant?

We've added a "ProcessDestroyer" after the 1.4.1 release that tries to kill
forked processes when Ant gets aborted.  I think this should fix your problem,
if the shutdown hook inside the VM that is running Ant gets time to fire, that
is.

Using Ant 1.5 alpha (CVS) results in the termination of the forked process 
(which is better) but this does not resolve the core problem.  The forked 
process shutdownhook is still not triggered.

Just a note concerning Ant 1.5 status - when invoking <java fork="true" ...> the
1.5 version does handle cleanup of the forked process however the when using
<java jar="whatever.jar" fork="true" ...> the process is not cleaned-up and
continues to execute independently.

Does anyone has the slighest idea of what's going on here that avoid the 
shutdown hook to be triggered ? I can't find what could be the problem !

The last entry of Stephen also puzzles me, there is no difference in Ant 
between executing a java command with or without a jar, we are just building 
the command line.

Concerning the last entry of the process continuing to run with -jar, it does 
not happen under JDK 1.3.1 but only with JDK 1.4

5003 seems like it could be related, but I've always viewed this specific 
problem as a black hole of hell and wanted to stay as far away from the even 
horizon as I could.

8510 looks like it could be solved by this patch, however.

With regards to:

> Does anyone has the slighest idea of 
> what's going on here that avoid the 
> shutdown hook to be triggered ? I can't 
> find what could be the problem !

The problem is that when the InterruptedException is caught and ignored in the 
waitFor() method, the executing process is assumed to have completed and is 
removed from the shutdown hook.  This is not the case when the thread is 
interrupted.  The issue is that catching the InterruptedException, and 
silently ignoring it, is acting just like the process exited normally.  The 
execute() method has no way of knowing if the thread was interrupted - and 
therefore the process is still executing - of if the process exited.  So when 
the InterruptedException is thrown in the thread, and the waitFor() exits, it 
just removes the shutdown hook and assumes the process is done.  Therefore the 
process will just happily continue to execute and no one will ever kill it.

Shoot.  Wrong bug.  Sorry for the noise.  I *thought* I was responding to bug 
10345.

*** Bug 12796 has been marked as a duplicate of this bug. ***

I did some tests concerning the shutdown hook problem with jdk 1.4. 
Here are the results:

shutdown hook works fine with:

jdk 1.3.x + application + no ant + windows (nt 4.0 + 2000)
jdk 1.3.x + application + no ant + linux
jdk 1.4.x + application + no ant + windows (nt 4.0 + 2000)
jdk 1.4.x + application + no ant + linux
jdk 1.3.x + application + ant 1.5.x (java + fork) + windows (nt 4.0 + 2000)
jdk 1.3.x + application + ant 1.5.x (java + fork) + linux
jdk 1.4.x + application + ant 1.5.x (java + fork) + linux

shutdown hook does not work with:

jdk 1.4.x + application + ant 1.5.x (java + fork) + windows (nt 4.0 + 2000)

I tried several jdk versions (1.3.0, 1.3.1, 1.4.0, 1.4.1) and ant versions
(1.5.0, 1.5.1). Always with the result described above.

So I think they must have changed something in jdk 1.4.x (win) about the
shutdown (hook) procedure, which is not integrated in ant so far. I don't think
it is a bug in the jdk, because when I run the application without ant all works
fine.

Maybe one of you can verify my results and I hope this will help to solve the
problem.

I make some experiments and make a "guess".
Maybe the guess is wrong, but hope the other information
will help to solve the problem.


 == Experiment 1: use JDK to invoke MyProg ==

"MyProg" is a program with shutdown hook installed.
Press "Ctrl+C" to stop it.

"Yes" means the showdown hook of "MyProg" is invoked.

java.exe MyProg
    JDK 1.3.1 Yes      (1.3.1_03-b03)
    JDK 1.4.0 Yes      (1.4.0-b92)


 == Experiment 2: use Ant 1.5.1 to invoke MyProg ==

Run "MyProg" with Ant. Press "Ctrl+C" to stop it.

Ant 1.5.1 has its own shotdown hook (added by
ProcessDestroyer.java) which call java.lang.Process.destroy()
on the process of "MyProg".

"Yes" means the showdown hook of "MyProg" is invoked.

Ant 1.5.1 -> MyProg
    JDK 1.3.1 *Yes*
    JDK 1.4.0  No


 == Experiment 3: use "Invoker" to invoke MyProg ==

"Invoker" is a program which launches "MyProg" with
System.exec().
"Invoker" has its own shotdown hook which call
java.lang.Process.destroy() on the process of "MyProg".

"Yes" means the showdown hook of "MyProg" been invoked.

java.exe Invoker -> MyProg
    JDK 1.3.1       JDK 1.4.0
        0  No           0  No
        1  No           1  No
        2 *Yes*         2  No

    case 0: Invoker call proc.destroy() directly
            (not in its shutdown hook)
    case 1: Invoker call proc.destroy() in its own
            shutdown hook triggered when the
            "main" thread completed.
    case 2: Invoker call proc.destroy() in its own
            shutdown hook triggered by Ctrl+C.


 == Code and Document ==

On Win32 systems java.lang.Process.destroy()
make system call TerminateProcess() in both
JDK 1.3.1 and 1.4.0 which unconditionally
cause a process to exit.
(src\win32\native\java\lang\Win32Process_md.c)

In the JavaDoc of java.lang.Runtime,
  "In rare circumstances the virtual machine may abort,
   that is, stop running without shutting down cleanly.
   This occurs when the virtual machine is terminated
   externally, for example with the SIGKILL signal on
   Unix or the TerminateProcess call on Win32.".


 == My guess ==

Process.destroy() which calls TerminateProcess() should
give "MyProg" no chance to run its shutdown hook.
There are two exceptions: (marked with '*')

    JDK 1.3.1 + Ant 1.5.1 + MyProg (with Ctrl+C)
    JDK 1.3.1 + Invoker + MyProg (with Ctrl+C)

I "guess" that JDK 1.3.1 also send "Ctrl+C" to MyProg
which cause it to run the shutdown hook before killed
by TerminateProcess() from Ant and Invoker. But I have
no evidence.



 == Code: MyProg.java ==

import java.io.FileWriter;
import java.io.IOException;

public class MyProg
{
    public static void log(String s)
    {
        try
        {
            FileWriter fw = new FileWriter("MyProg.txt", true);
            fw.write(s + "\n");
            fw.flush();
            fw.close();
        } catch (IOException e) { }
    }

    public static void main(String[] args) throws Exception
    {
        log("MyProg: main() : " + System.getProperty("java.runtime.version"));
        Runtime.getRuntime().addShutdownHook(new Thread() {
            public void run()
            {
                log("MyProg: hook invoked");
            }});
        log("MyProg: hook installed");
        while (true) ;
    }

}


 == Code: Invoker.java ==

public class Invoker
{
    public static Process exec()
    {
        try
        {
            System.out.println("before exec");
            Process proc = Runtime.getRuntime().exec("java MyProg");
            System.out.println("after exec, sleep 5 sec");
            Thread.currentThread().sleep(5 * 1000);
            System.out.println("ready");
            return proc;
        } catch (Exception e) { e.printStackTrace(); return null; }
    }

    public static void addHook(final Process proc)
    {
        System.out.println("enter addHook");
        Runtime.getRuntime().addShutdownHook(new Thread() {
            public void run()
            {
                System.out.println("Invoker: hook invoked");
                System.out.println("before destory");
                proc.destroy();
                System.out.println("after destory");
            }});
        System.out.println("leave addHook");
    }

    public static void destroyDirectly()
    {
        Process proc = exec();
        System.out.println("before destroy");
        proc.destroy();
        System.out.println("after destroy");
    }

    public static void destroyFromHook1()
    {
        Process proc = exec();
        addHook(proc);
    }

    public static void destroyFromHook2()
    {
        Process proc = exec();
        addHook(proc);
        System.out.println("waiting for ctrl+c");
        while (true) ;
    }


    public static void main(String[] args) throws Exception
    {
        switch (Integer.parseInt(args[0]))
        {
            case 0 :
                destroyDirectly();
                break;
            case 1 :
                destroyFromHook1();
                break;
            case 2 :
                destroyFromHook2();
                break;

        }
    }
}

 == build.xml ==
 
<project name="A" default="A" basedir=".">
    <target name="A">
        <java classname="MyProg" fork="yes">
            <classpath>
                <pathelement location="."/>
            </classpath>
        </java>
    </target>
</project> 

If 1.4.x really does use ::TerminateProcess() then that is kind of brutal and
potentially dirty. I wonder if we can do some very low level stuff to work out
exactly what goes on.

Both JDK 1.3.? (I don't know the exact version) and JDK 1.4.0
use the same code to destroy() a process.

JNIEXPORT void JNICALL
Java_java_lang_Win32Process_destroy(JNIEnv *env, jobject process)
{
    jboolean exc;
    jint handle = JNU_GetFieldByName(env, &exc, process, "handle", "I").i;
    if (exc) {
        return;
    }
    TerminateProcess((void *)handle, 1);
}

Maybe sometimes it returns before invoking ::TerminateProcess() ?

P.S. Is it legal to post the code here?

I have resolved some aspects of this bug, although it mainly applies to Unix
systems (more on that in a minute).

Under Ant 1.5 under Linux, the ProcessDestroyer does not wait for the
terminating processes to complete their shutdowns. If they have anything more
than a trivial shutdown process, it would get truncated as the Ant VM shuts
down. The ProcessDestroyer also blocked Execute.java in the remove() method as
it held the lock on the processes object. So even if the process managed to
generate output, it may not have made it to the Ant output log.

Under Windows and JDK 1.4, the process shutdown hooks are not fired. This is
marked by Sun as "not a bug". See
http://developer.java.sun.com/developer/bugParade/bugs/4485742.html
although http://developer.java.sun.com/developer/bugParade/bugs/4671966.html may
be able to change that.

JDK 1.3 appears to fire the shutdown hook on Windows but it is something of an
illusion as the shutdown is not allowed to complete. If you put this in the
given Demo class' run method

                System.err.println("DEMO shutdown hook fired");
                try {
                    PrintWriter pw
                        = new PrintWriter(new FileOutputStream("test.txt"));
                    Thread.currentThread().sleep(3000);
                    pw.println("Test");
                    pw.close();
                } catch (Exception e) {
                }
                System.err.flush();

you will find that test.txt is created but no content is written. This works
under Linux.

Note that the use of output or error streams to determine if the shutdown hook
has fired or not is suspect. The above file based approach is more conclusive.
This is because the output generated may still be within a couple of Ant stream
processing threads (daemons) which get terminated in the shutdown process.

The Main.java does not really demonstrate a problem. It gives no info about the
firing of the shutdown hook in the subprocess. In fact it has the same behaviour
as Ant - the subprocess shutdown is not fired. It is indeed a JVM bug^H^H^Hfeature.

I'm marking as fixed from Ant's point of view. You need to lobby Sun to address
the shutdown issues under windows.

*** Bug 22026 has been marked as a duplicate of this bug. ***

I am using ant 1.6.2 and have a working test case on MacOSX that clearly shows that a shutdown hook 
is never invoked when you ctrl-c an app run from ant's java task.

Created attachment 13672 [details]
a small simple test case that shows the bug in action

see the readme in the zip file.

I tested your example problem and saw the same thing under linux.
However, I then read Conor's report

"Note that the use of output or error streams to determine if the shutdown hook
has fired or not is suspect. The above file based approach is more conclusive."

So, I changed the code to do:
	public class Shutter extends Thread {
		public void run() {
//			log.debug("clean shutdown");
                    System.err.println("DEMO shutdown hook fired");
                    try {
                        java.io.PrintWriter pw
                            = new java.io.PrintWriter(
                      new java.io.FileOutputStream("test.txt"));
                        Thread.currentThread().sleep(3000);
                        pw.println("Test");
                        pw.close();
                    } catch (Exception e) {
                    }
                    System.err.flush();
                    System.out.println("clean shutdown...");
		}
	}

And noted that clean shutdown... was not seen, but the file was created
and written to, and the DEMO shutdown hook fired message was seen.
Can you make the above changes and see if they fix the problem?

(In reply to comment #20)
Interesting.  I made the changes you described and lo - the file is written but i get no consol output at 
all.
The reason i first came across this bug is that a project I am woring on opens a connection to an SMPP 
server and the shutdown hook is supposed to close that connection if the user ctrl-c's the app.  this 
works fine if i run it from a shell script but not from ant.
I can see from my SMPP server logs that a connection is still left there dangling.
Is there some work-around for this?

Created attachment 13734 [details]
An update to the test case that shows the dehaviour described by peter reilly

this works but raises more questions - ie what can and can not be expected to
execute in a shutdown hook when you expect the app to be run from ant and the
user is able to ctrl-c.
perhaps the ant docs need to be updated to reflect this behaviour.

The writing to std output/error for forked processes are handled by
thread objects reading the relativent streams. At shutdown time
these threads may be stopped before they have finished reading their
stream - so it is a timing and flushing behaviour dependent what
is actually written to the console.

This timing issue should not effect closing the connection to an SMPP
server. Maybe you could use the write to file techinque to debug your
problem. (Surely the connection should be closed automatticlly by the
OS when the ant process ends?)

Closing it since the issues cited by Conor really point at a JDK not-a-bug.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4671966 is still not closed AFAICS.