Harmony
  1. Harmony
  2. HARMONY-148

java.nio.charset.CharsetDecoder: decode(in,out,endOfInput) method doesn't preserve replace string for successive decode invocation while "out" doesn't have engouh space for replace string.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Classlib
    • Labels:
      None

      Description

      Consider following senario:
      During the invocation of method java.nio.charset.CharsetDecoder.decode(in,out,false), method "decodeLoop" returns malformed CoderResult, and the action is set as "CodingErrorAction.REPLACE". So replace action should be taken, but "out" doesn't hold enough space for the replace string. Therefore, CoderResult.OVERFLOW should be returned. Harmony does return OVERFLOW currently, but the replace string won't put into new allocated output any more. In other words, the replace string is missing from the view of "stream decoding". Therefore, replace string should be preserved for successive decode(in,out,endOfInput) invacation.
      Both RI5.0 and Harmony fails on the following test case. However, according to the spec, we think it is a defect of RI5.0.

      ================================== Test Case======================
      /*

      • Test the method decode(ByteBuffer) .
        */
        public void testDecode_ReplaceOverflow() throws Exception {
        String replaceString = "a";
        Charset cs = Charset.forName("UTF-8");
        MockMalformedDecoder decoder = new MockMalformedDecoder(cs);
        decoder.onMalformedInput(CodingErrorAction.REPLACE);
        decoder.replaceWith(replaceString);
        CharBuffer out = CharBuffer.allocate(1);
        // MockMalformedDecoder treats the second byte '0x38' as malformed,
        // but "out" doesn't have enough space for replace string.
        ByteBuffer in = ByteBuffer.wrap(new byte[] {0x45,0x38,0x45,0x45}

        );
        CoderResult result = decoder.decode(in,out,false);
        assertTrue(result.isOverflow());

      // allocate enough space for "out"
      out = CharBuffer.allocate(10);
      // replace string should be put into "out" firstly,
      // and then decode "in".
      result =decoder.decode(in,out,true);
      out.flip();
      assertTrue(result.isUnderflow());
      assertEquals("abb",out.toString());
      }
      /*

      • Mock decoder. It treats byte whose value is less than "0x40" as malformed.
        */
        static class MockMalformedDecoder extends java.nio.charset.CharsetDecoder {

      public MockMalformedDecoder(Charset cs)

      { super(cs, 1, 10); }

      /*

      • It treats byte whose value is less than "0x40" as malformed.
      • Otherwise, it's decoded as 'b'.
        */
        protected CoderResult decodeLoop(ByteBuffer in, CharBuffer out) {
        System.out.println("Using MockMalformedDecoder");
        while (in.hasRemaining())
        Unknown macro: { byte b = in.get(); if(b < 0x40){ return CoderResult.malformedForLength(1); } if(!out.hasRemaining()){ return CoderResult.OVERFLOW; } out.put((char)'b'); }

        return CoderResult.UNDERFLOW;
        }
        }

      1. CharsetDecoderTest_patch_148.txt
        2 kB
        Paulex Yang
      2. CharsetDecoder_patch_148.txt
        1 kB
        Richard Liang

        Activity

        Richard Liang created issue -
        Hide
        Richard Liang added a comment -

        This patch fixes bug Harmony 148. Would you please have a try ?
        The patch could be simply applied by clicking "team"->"apply patch...".
        Thank you very much!

        Show
        Richard Liang added a comment - This patch fixes bug Harmony 148. Would you please have a try ? The patch could be simply applied by clicking "team"->"apply patch...". Thank you very much!
        Richard Liang made changes -
        Field Original Value New Value
        Attachment CharsetDecoder_patch_148.txt [ 12323655 ]
        Tim Ellison made changes -
        Assignee Tim Ellison [ tellison ]
        Tim Ellison made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Tim Ellison added a comment -

        Richard,

        Fix applied to NIO_CHAR module java.nio.charset.CharsetDecoder at repo revision 383862.

        Please can you provide your tests as a patch? thanks.

        Show
        Tim Ellison added a comment - Richard, Fix applied to NIO_CHAR module java.nio.charset.CharsetDecoder at repo revision 383862. Please can you provide your tests as a patch? thanks.
        Hide
        Paulex Yang added a comment -

        The test patch is attached. thx.

        Show
        Paulex Yang added a comment - The test patch is attached. thx.
        Paulex Yang made changes -
        Attachment CharsetDecoderTest_patch_148.txt [ 12323897 ]
        Hide
        Tim Ellison added a comment -

        Paulex,

        Applied the test case patch. The tests fail on the reference implementation as expected (we will add it to the reference impl exclusion list one we have that in place).

        Please check that the patch was applied as you expected.

        Show
        Tim Ellison added a comment - Paulex, Applied the test case patch. The tests fail on the reference implementation as expected (we will add it to the reference impl exclusion list one we have that in place). Please check that the patch was applied as you expected.
        Tim Ellison made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Paulex Yang added a comment -

        It looks good. thank you, Tim.

        Show
        Paulex Yang added a comment - It looks good. thank you, Tim.
        Hide
        Tim Ellison added a comment -

        Verified by Paulex.

        Show
        Tim Ellison added a comment - Verified by Paulex.
        Tim Ellison made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Tim Ellison
            Reporter:
            Richard Liang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development