Bug 11357 - ASCII85Filter.convertWord() method does not work properly within specific environment
Summary: ASCII85Filter.convertWord() method does not work properly within specific env...
Status: CLOSED WONTFIX
Alias: None
Product: Fop - Now in Jira
Classification: Unclassified
Component: pdf (show other bugs)
Version: all
Hardware: All Linux
: P3 blocker
Target Milestone: ---
Assignee: fop-dev
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-08-01 06:56 UTC by Dipl.-Ing. Ralf Pichler
Modified: 2012-04-01 13:53 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dipl.-Ing. Ralf Pichler 2002-08-01 06:56:42 UTC
The following bug occurs only within the following test environment:
OS Linux SuSE 7.3 (i386) Kernel version 2.4.10
JRE 1.3.1 Standard Edition, Classic VM (build 1.3.1, J2RE 1.3.1 IBM build 
cxia32131w-20020410 ORB130)

The bug (which is located within the JVM but can be solved by changing the 
method ASCII85Filter.convertWord()) causes the stated method to calculate 
invalid valus for c1 - c5, and a number of "illegal char value " texts are 
written to the screen.
The reason is that the IBM JVM has (sometimes!, not always) problems with long 
values between [0x7FFF FFFF .. 0xFFFF FFFF]. As a result the line
byte c1 = (byte)((word / base85_1) & 0xFF);
returns negative values for c1 (word and base85_1 are positive of course).
As mentioned, this error does not occur always, even with the same number. It 
also depends on the position of the number within the byte array passed to 
encode().

To solve this problem I changed the ASCII85Filter.convertWord() method:

  private static final double dbase85_4 = (double)85;
  private static final double dbase85_3 = base85_4 * base85_4;
  private static final double dbase85_2 = base85_3 * base85_4;
  private static final double dbase85_1 = base85_2 * base85_4;

  private static byte[] convertWord(long word) 
  {
    byte[] ret = null;

    word = word & 0xffffffffL;
    if (word < 0)
      word = -word;

    if (word == 0) 
    {
      byte[] result = {(byte)ASCII85_ZERO};
      return result;
    }
 
    byte c1 = 0;
    byte c2 = 0;
    byte c3 = 0;
    byte c4 = 0;
    byte c5 = 0;

// Use a different method to calculate c1-c5 if word is >= 2**31
    if (word > 0x7fffffffL)
    {
      double dword = (double)word;
      c1 = (byte)(((long)(dword / base85_1)) & 0xFF);
      double d1 = (double)c1 * dbase85_1;
      c2 = (byte)(((long)((dword - d1) / base85_2)) & 0xFF);
      d1 = d1 + ((double)c2 * dbase85_2);
      c3 = (byte)(((long)((dword - d1) / base85_3)) & 0xFF);
      d1 = d1 + ((double)c3 * dbase85_3);
      c4 = (byte)(((long)((dword - d1) / base85_4)) & 0xFF);
      d1 = d1 + ((double)c4 * dbase85_4);
      c5 = (byte)(((long)(dword - d1)) & 0xFF);
    }
    else
    {
      c1 = (byte)((word / base85_1) & 0xFF);
      long l1 = c1 * base85_1;
      c2 = (byte)(((word - l1) / base85_2) & 0xFF);
      l1 = l1 + (c2 * base85_2);
      c3 = (byte)(((word - l1) / base85_3) & 0xFF);
      l1 = l1 + (c3 * base85_3);
      c4 = (byte)(((word - l1) / base85_4) & 0xFF);
      l1 = l1 + (c4 * base85_4);
      c5 = (byte)(((word - l1)) & 0xFF);
    }

    ret = new byte[] {(byte)(c1 + ASCII85_START), (byte)(c2 + ASCII85_START),
                      (byte)(c3 + ASCII85_START), (byte)(c4 + ASCII85_START),
                      (byte)(c5 + ASCII85_START)};
    for (int i = 0; i < ret.length; i++)
    {
      if (ret[i] < 33 || ret[i] > 117) 
      {
        System.out.println("illegal char value " + new Integer(ret[i]));
      }
    }
    return ret;
  }

Some additional notes:
* I think this bug also explains the bug report 11277
* I wonder why this method contains
    word = word & 0xffffffffL;
    if (word < 0)
      word = -word;
which I did not change, however if everything would work as expected you might 
think that this lines are senseless.
* For critical numbers (bit 31 is 1) the calculation of c1-c5 is done using 
double values instead of long. For smaller numbers the original algorithm is 
used. I removed the duplicate sub-calculations by introducing the d1/l1 
variables, maybe this speeds up performance a bit.
Comment 1 J.Pietschmann 2002-08-01 20:46:35 UTC
I committed a simplified version of the conversion code to the maintenance
branch, which will hopefully circumvent the JVM bug.
Anyway, you should get the JVM bug fixed.
Comment 2 Glenn Adams 2012-04-01 13:53:20 UTC
batch transition to closed remaining pre-FOP1.0 resolved bugs