[DAFFODIL-219] implement utf-8 variable width characters

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: s3
• Component/s: Back End
• Assignee: Paul Chu (Inactive)
• Reporter: Mike Beckerle
• Created: 29/Jun/12 18:10
• Updated: 31/Jul/12 20:46
• Resolved: 13/Jul/12 14:08

Description

If we implement utf-8 (variable width encoding), then many of the IBM tests will work. (Well, at least the 5 categorized as IBMTestsOnNumbers will work (I know because changing that test to use encoding="ascii" and they all run. But,... we don't want to change the IBM tests.)

Implementing utf-8 is a matter of implementing StringFixedLengthInBytesVariableWidthCharacters, and StringFixedLengthInVariableWidthCharacters.

These are variations on the theme established by StringFixedLengthInBytes, which is our current workhorse.

I suggest we fix this in s3 because I don't think it's terribly difficult, so the effort to accomplishment ratio is high