Uploaded image for project: 'XMLGraphicsCommons'
  1. XMLGraphicsCommons
  2. XGC-93

Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 2.2
    • Component/s: postscript
    • Environment:

      Description

      The method org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(OutputStream) from xmlgraphics-commons-1.5.jar calls the method DeflaterOutputStream.write(byte) byte by byte (see lines 242 - 244):

      private boolean optimizedWriteTo(OutputStream out)
      throws IOException {
      if (this.firstTileDump) {
      Raster raster = image.getTile(0, 0);
      DataBuffer buffer = raster.getDataBuffer();
      if (buffer instanceof DataBufferByte) {
      byte[] bytes = ((DataBufferByte) buffer).getData();
      // see determineEncodingColorModel() to see why we permute B and R here
      if (isBGR) {
      for (int i = 0; i < bytes.length; i += 3)

      { out.write(bytes[i + 2]); // 242 out.write(bytes[i + 1]); // 243 out.write(bytes[i]); // 244 }

      } else

      { out.write(bytes); }
      return true;
      }
      }
      return false;
      }

      Under WebSphere, in a multi-session environment, this leads to warnings about possibly hanging threads:

      [07.03.13 08:34:22:673 MEZ] 0000000d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1303" (00000547) has been active for 165445 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
      at java.util.zip.Deflater.deflateBytes(Native Method)
      at java.util.zip.Deflater.deflate(Deflater.java:306)
      at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
      at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
      at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
      at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:242)
      ...
      [07.03.13 08:38:18:886 MEZ] 00000547 ThreadMonitor W WSVR0606W: Thread "WebContainer : 1303" (00000547) was previously reported to be hung but has completed. It was active for approximately 401662 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung.
      [07.03.13 08:40:22:723 MEZ] 0000002d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1323" (0000055b) has been active for 221920 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
      at java.util.zip.Deflater.deflateBytes(Native Method)
      at java.util.zip.Deflater.deflate(Deflater.java:306)
      at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
      at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
      at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
      at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:244)
      ...

      The threads will finish their work, but this takes a long time.

      The reason for this is, that the number n of calls to java.util.zip.DeflaterOutputStream.write(byte) might be in a hight range. I have seen cases, where n=4094496. This leads to n calls of DeflaterOutputSteam.write(byte[] b, int off, int len), where off=0 and len=1:

      public void write(byte[] b, int off, int len) throws IOException {
      if (def.finished()) { throw new IOException("write beyond end of stream"); }
      if ((off | len | (off + len) | (b.length - (off + len))) < 0) { throw new IndexOutOfBoundsException(); } else if (len == 0) { return; }
      if (!def.finished()) {
      // Deflate no more than stride bytes at a time. This avoids
      // excess copying in deflateBytes (see Deflater.c)
      int stride = buf.length;
      for (int i = 0; i < len; i+= stride) {
      def.setInput(b, off + i, Math.min(stride, len - i)); // line 116
      while (!def.needsInput()) { deflate(); // line 118 }
      }
      }
      }

      Since java.util.zip.Deflater internally uses synchronized blocks, esspecially in setInput(byte[], int, int) and deflate(byte[], int, int), one should reduce the number of calls to these methods as much as possible.

      The number of calls of Deflater.setInput(byte[] b, int off, int len) in line 116
      above could, e.g., be reduced by the factor 512 (=buffer.length), if the method ImageEncodingHelper.optimizedWriteTo() would be written in this way:

      private boolean optimizedWriteTo(OutputStream out)
      throws IOException {
      if (this.firstTileDump) {
      Raster raster = image.getTile(0, 0);
      DataBuffer buffer = raster.getDataBuffer();
      if (buffer instanceof DataBufferByte) {
      byte[] bytes = ((DataBufferByte) buffer).getData();
      // see determineEncodingColorModel() to see why we permute B and R here
      if (isBGR) {
      byte[] bytesPermutated = new byte[bytes.length];
      for (int i = 0; i < bytes.length; i += 3) { bytesPermutated[i] = bytes[i+2]; bytesPermutated[i+1] = bytes[i+1]; bytesPermutated[i+2] = bytes[i]; }
      out.write(bytesPermutated);
      } else { out.write(bytes); }

      return true;
      }
      }
      return false;
      }

      In this version, n calls of DeflaterOutputStream.write(byte) are replaced by one call of DeflaterOutputStream(byte[]), which is much faster and offers less blocking potential.

      Without this optimization and with activated zlib compression in FOP 1.1, we always got warnings from WeSphere's thread monitor, that some threads may hang.

        Attachments

        1. xgc-93.patch
          1 kB
          Matthias Reischenbacher

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              aklm Andre Klemann
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: