Harmony
  1. Harmony
  2. HARMONY-67

java.nio.charset.Charset.decode(ByteBuffer) throws unexpected BufferOverflowException for UTF-16BE, UTF-16LE, UTF-16 charsets.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Classlib
    • Labels:
      None

      Description

      According to j2se 1.4.2 specification for Charset.decode(ByteBuffer b) the method must not throw any exceptions.
      The test listed below shows that there is unexpected BufferOverflowException thrown if charset name is one in the following: UTF-16BE, UTF-16LE, UTF-16.
      BEA does not throw any exceptions.

      Code to reproduce:
      import java.nio.charset.Charset;
      import java.nio.ByteBuffer;
      import java.nio.CharBuffer;
      public class test2 {
      public static void main(String[] args) throws Exception {
      byte[] b = new byte[]

      {(byte)1};
      ByteBuffer buf= ByteBuffer.wrap(b);
      CharBuffer charbuf=Charset.forName("UTF-16").decode(buf);
      System.out.println("CharBuffer.length()="+ charbuf.length());
      }
      }

      Steps to Reproduce:
      1. Build Harmony (check-out on 2006-01-30) j2se subset as described in README.txt.
      2. Compile test2.java using BEA 1.4 javac
      > javac -d . test2.java
      3. Run java using compatible VM (J9)
      > java -showversion test2

      Output:
      C:\tmp>C:\jrockit-j2sdk1.4.2_04\bin\java.exe -showversion test2
      java version "1.4.2_04"
      Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_04-b05)
      BEA WebLogic JRockit(TM) 1.4.2_04 JVM (build ari-31788-20040616-1132-win-ia32, Native Threads, GC strategy: parallel)
      CharBuffer.length()=0

      C:\tmp>C:\harmony\trunk\deploy\jre\bin\java -showversion test2
      (c) Copyright 1991, 2005 The Apache Software Foundation or its licensors, as applicable.
      java.nio.BufferOverflowException
      at java.nio.CharBuffer.put(CharBuffer.java:662)
      at java.nio.CharBuffer.put(CharBuffer.java:629)
      at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:406)
      at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:243)
      at java.nio.charset.Charset.decode(Charset.java:630)
      at test2.main(test2.java:8)

      Suggested junit test case:
      ------------------------ CharsetTest.java -------------------------------------------------
      import java.nio.charset.Charset;
      import java.nio.ByteBuffer;
      import java.nio.CharBuffer;

      import junit.framework.*;

      public class CharsetTest extends TestCase {
      public static void main(String[] args) { junit.textui.TestRunner.run(CharsetTest.class); }
      public void test_decode() {
      byte[] b = new byte[] {(byte)1}

      ;
      ByteBuffer buf= ByteBuffer.wrap(b);
      CharBuffer charbuf=Charset.forName("UTF-16").decode(buf);
      assertEquals("Assert 0: charset UTF-16",0,charbuf.length());

      charbuf=Charset.forName("UTF-16BE").decode(buf);
      assertEquals("Assert 1: charset UTF-16BE",0, charbuf.length());

      charbuf=Charset.forName("UTF-16LE").decode(buf);
      assertEquals("Assert 2: charset UTF16LE",0, charbuf.length());

      }
      }

        Issue Links

          Activity

          Svetlana Samoilenko created issue -
          Hide
          Vladimir Strigun added a comment -

          Looks like duplicate of Harmony-33

          Show
          Vladimir Strigun added a comment - Looks like duplicate of Harmony-33
          Tim Ellison made changes -
          Field Original Value New Value
          Link This issue duplicates HARMONY-33 [ HARMONY-33 ]
          Tim Ellison made changes -
          Assignee Tim Ellison [ tellison ]
          Tim Ellison made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Tim Ellison added a comment -

          Svetlana,

          Fixed in NIO_CHAR module java.nio.charset.CharsetDecoder at repo revision 377729.

          Please check that this fully resolves your problem.

          Show
          Tim Ellison added a comment - Svetlana, Fixed in NIO_CHAR module java.nio.charset.CharsetDecoder at repo revision 377729. Please check that this fully resolves your problem.
          Tim Ellison made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          Hide
          Svetlana Samoilenko added a comment -

          Tim, I confirm that there is no exeption any more, but the result of decode method is wrong.
          Harnony returns charbuf.length())==1 while BEA returns charbuf.length())==0.
          If to set UTF-8 charset, the result is equal.

          Show
          Svetlana Samoilenko added a comment - Tim, I confirm that there is no exeption any more, but the result of decode method is wrong. Harnony returns charbuf.length())==1 while BEA returns charbuf.length())==0. If to set UTF-8 charset, the result is equal.
          Tim Ellison made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Vladimir Strigun added a comment -

          I don't think that this bug should be reopened. Possibly it's another "compatibility" bug, and I'll try to explain why.
          Specification of decode method says that "A newly-allocated character buffer containing the result of the decoding operation." will be returned. Harmony implementation of decode buffer returns CharBuffer with length equal to 1. Inside resulting buffer we can see one element: "FFFD", i.e default replacement for unmappable character. In compliance with spec Charset.decode(buffer) method invoke following line:
          cs.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
          So, IMO Harmony implementation is fully correspond to the spec.

          Show
          Vladimir Strigun added a comment - I don't think that this bug should be reopened. Possibly it's another "compatibility" bug, and I'll try to explain why. Specification of decode method says that "A newly-allocated character buffer containing the result of the decoding operation." will be returned. Harmony implementation of decode buffer returns CharBuffer with length equal to 1. Inside resulting buffer we can see one element: "FFFD", i.e default replacement for unmappable character. In compliance with spec Charset.decode(buffer) method invoke following line: cs.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb); So, IMO Harmony implementation is fully correspond to the spec.
          Hide
          Tim Ellison added a comment -

          Svetlana, can you explain why you think the decode behavior is wrong?

          The test, as written in our SVN repository, passes on Harmony code and the Sun implementation of Java.
          As Vladimir wrote, this may be a compatibility issue with the BEA implementation, and I agree that our behavior is to spec.

          Show
          Tim Ellison added a comment - Svetlana, can you explain why you think the decode behavior is wrong? The test, as written in our SVN repository, passes on Harmony code and the Sun implementation of Java. As Vladimir wrote, this may be a compatibility issue with the BEA implementation, and I agree that our behavior is to spec.
          Hide
          Svetlana Samoilenko added a comment -

          Tim,
          I agree with Vladimir,
          this condition cs.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb) is correct, it is compatibility issue.

          Show
          Svetlana Samoilenko added a comment - Tim, I agree with Vladimir, this condition cs.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb) is correct, it is compatibility issue.
          Hide
          Tim Ellison added a comment -

          Verified by Svetlana.
          (If anyone disagrees I'm happy to reopen it)

          Show
          Tim Ellison added a comment - Verified by Svetlana. (If anyone disagrees I'm happy to reopen it)
          Tim Ellison made changes -
          Status Reopened [ 4 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          13d 1h 21m 1 Tim Ellison 14/Feb/06 23:00
          In Progress In Progress Resolved Resolved
          3m 58s 1 Tim Ellison 14/Feb/06 23:04
          Resolved Resolved Reopened Reopened
          20h 3m 1 Tim Ellison 15/Feb/06 19:07
          Reopened Reopened Closed Closed
          2d 1h 16m 1 Tim Ellison 17/Feb/06 20:23

            People

            • Assignee:
              Tim Ellison
              Reporter:
              Svetlana Samoilenko
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development