I've attached a patch that addresses this bug and another I discovered. First, with regard to the issue at hand, it seems to me that an empty string (len == 0) should be "transcoded", with the result being another zero-terminated empty string. Otherwise the caller has an undue burden to examine the string before attempting to transcode it. Also, the Throw at line 624 is warranted, in case the input XMLCh string is malformed (in my book, that includes having a premature zero before len characters). So, I avoid an early exit. Instead, I add enough space to allocSize for the 4 terminating zeroes, which has two beneficial effects – in some cases it avoids a reallocation, and it also guarantees enough space for at least one UTF-8 transcoded character, so we can safely keep the Throw. However, if the input string is empty, we just skip calling transcodeTo().
I applied a similar fix to TranscodeFromStr::transcode(), and that's where I found an entirely different bug. When it needs to reallocate, it does a memcpy(newBuf, fString, fCharsWritten) to copy the existing partial string to the new, larger buffer. However, memcpy() takes a count in units of bytes, while fCharsWritten is a count of XMLCh! The call should be memcpy(newBuf, fString, fCharsWritten * sizeof(XMLCh)).
I made a couple of other minor changes to improve readability and optimize.