[GROOVY-6852] groovy.json.JsonOutput throws ArrayIndexOutOfBoundsException when serializing some Strings that contain a high percentage of non-ASCII characters.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.2, 2.4.0-beta-1
• Fix Version/s: 2.3.5, 2.4.0-beta-2
• Component/s: command line processing, JSON
• Environment: JDK 1.7.0_60
• Assignee: Guillaume Sauthier
• Reporter: Lukas Loehrer
• Created: 06/Jun/14 13:32
• Updated: 26/Jul/14 01:44
• Resolved: 25/Jul/14 03:29

Description

Since Groovy 2.3.0, groovy.json.JsonOutput sometimes throws
java.lang.ArrayIndexOutOfBoundsException when serializing Strings that
contain a large percentage of non-ASCII characters. The problem seems
to occur because the method
groovy.json.internal.CharBuf.doAddJsonEscapedString does not always
grow the output buffer appropriately. At the beginning, the method
ensures that there is at least 2*<input_length>+2 characters of free
space in the output buffer assuming optimistically that the majority
of characters will not have to be escaped. Therefore, it is necessary
to check if there is enough free space in the buffer before adding
each character to the buffer. There appear to be the following
problems in the current implementation:

• The code currently only checks the free space in the buffer before adding json-control characters and other special characters that must be escaped. The check should also be performed before adding "normal" characters.
• I think there is an off-by-one error in the check for available space that is done before adding a character that must be escaped. The check ensures only for 5 available characters of free space, instead of 6.
• It is necessary to make sure there is always space for the closing double-quote character at the end.

The following Example Groovy shell session demonstrates the cases:

Groovy Shell (2.3.2, JVM: 1.7.0_60)
Type ':help' or ':h' for help.
-------------------------------------------------------------------------------
// Adding a "normal" character fails
groovy:000> groovy.json.JsonOutput.toJson('12ööööö12')
ERROR java.lang.ArrayIndexOutOfBoundsException:
33
        at groovy.json.internal.CharBuf.doAddJsonEscapedString (CharBuf.java:525)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:394)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:357)
        at groovy.json.JsonOutput.writeCharSequence (JsonOutput.java:309)
        at groovy.json.JsonOutput.toJson (JsonOutput.java:97)
        at groovy.json.JsonOutput$toJson.call (Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall (CallSiteArray.java:45)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call (AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call (AbstractCallSite.java:116)

// Exception when writing a "special" character to the buffer
groovy:000> groovy.json.JsonOutput.toJson('öö')
ERROR java.lang.ArrayIndexOutOfBoundsException:
12
        at groovy.json.internal.CharBuf.doAddJsonEscapedString (CharBuf.java:505)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:394)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:357)
        at groovy.json.JsonOutput.writeCharSequence (JsonOutput.java:309)
        at groovy.json.JsonOutput.toJson (JsonOutput.java:97)
...

// In the following example, adding the final double-quote character fails:
groovy:000> groovy.json.JsonOutput.toJson('12ööö')
ERROR java.lang.ArrayIndexOutOfBoundsException:
21
        at groovy.json.internal.CharBuf.doAddJsonEscapedString (CharBuf.java:533)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:394)
        at groovy.json.internal.CharBuf.addJsonEscapedString (CharBuf.java:357)
        at groovy.json.JsonOutput.writeCharSequence (JsonOutput.java:309)
        at groovy.json.JsonOutput.toJson (JsonOutput.java:97)
...