Uploaded image for project: 'Groovy'
  1. Groovy
  2. GROOVY-3275

Process.consumeProcessOutput leads to a race condition

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.5.6
    • 1.6.5, 1.7-beta-2
    • groovy-jdk
    • None
    • Linux (RHEL) x64, JDK 1.5

    Description

      Groovy provides convenience methods in Process like consumeProcessOutput that spawns off threads to handle an external process' I/O. Unfortunately, these threads are sort of dumped on the floor, leaving the user with no way to manage race conditions where threads finish at different times. For example, the following code tries to capture stdout into a compressed file:

      Process p = ['mysqldump', 'big_database'].execute()
      def os = new GZIPOutputStream(new File('dbdump.sql.gz').newOutputStream())
      p.consumeProcessOutput os, System.err
      p.waitFor()
      os.close()

      This can result in the following exception. This is because while the process itself may terminate after Process.waitFor(), the thread writing the corresponding output stream may not be finished writing yet. So if I call OutputStream.close() immediately after Process.waitFor(), there is a chance that the gzip stream may not be fully written yet, so the consumer thread tries to write to an output stream that has already been closed. The trouble is, there is no way for me to tell when the consumer threads are done processing.

      Exception in thread "Thread-27" groovy.lang.GroovyRuntimeException: exception while dumping process stream
      at org.codehaus.groovy.runtime.DefaultGroovyMethods$ByteDumper.run(DefaultGroovyMethods.java:10063)
      at java.lang.Thread.run(Thread.java:595)
      Caused by: java.io.IOException: write beyond end of stream
      at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:104)
      at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)
      at org.codehaus.groovy.runtime.DefaultGroovyMethods$ByteDumper.run(DefaultGroovyMethods.java:10060)
      ... 1 more

      There are two possible solutions to this:
      1. Process.waitFor should also wait for the consumer threads to finish.
      2. Find a way to expose the consumer thread objects so the user can call Thread.join() on them.

      Attachments

        1. gvy.patch
          6 kB
          Jim Jagielski
        2. gvy.patch
          2 kB
          Jim Jagielski

        Activity

          People

            paulk Paul King
            cwong15 Christopher Wong
            Votes:
            3 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: