Streaming Base64

• Key: CODEC-69
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Jochen Wiedmann
• Reporter: Julius Davies
• Fix Version/s: 1.4

Description

Would be nice to have Base64InputStream and Base64OutputStream, with both supporting ENCODE or DECODE modes.

Comments

[Julius Davies]

Patch introduces Base64InputStream and Base64OutputStream. They wrap any pre-existing InputStream or OutputStream, like so:

InputStream in = new FileInputStream("/path/to/file.base64");
in = new Base64InputStream(in, false); // false == DECODE

The guts of Base64 have been ripped out and rearranged to better suit this way of working, but all the JUnit tests still pass!

Thanks to "commons" project in ws.apache.org for much of this code.
http://svn.apache.org/repos/asf/webservices/commons/trunk/modules/util/

[Julius Davies]

Related to CODEC-8, but I wasn't brave enough to try a general approach, and I need streaming Base64 urgently! (500MB+ files sometimes encrypted/decrypted in not-yet-commons-ssl!)

[Jochen Wiedmann]

Applied. There is, of course, CODEC-8, but IMO this is so far the best piece of code that commons-codec has seen, so that we may think about the API later on.

[Julius Davies]

Thanks so much for accepting this patch!

I noticed some minor typos in some of my javadocs, and a spurious (byte) cast.... I'll probably upload an additional patch in a few days.

[Julius Davies]

Oops - Base64 constructors are not public. In previous version there was no declared constructor, which is identical to having "public Base64() {}".

The constructors need to be public to support anyone who is using the "BinaryEncoder" or "BinaryDecoder" interfaces instead of using Base64 directly.

[Julius Davies]

To reply to Sebb's comment on CODEC-8:

Base64 encoding will only emit bytes in chunks of 4.

For example:

1. read a byte... nothing emitted
2. read a byte... nothing emitted
3. read a byte... 4 bytes emitted.
4. read a byte... nothing emitted.
5. EOF! 4 bytes emitted (e.g. AA==).

So even if a "lineLength" of 75 is passed to the constructor, the lines will still be 76 characters long. Not because we ever actually round the number up, but because of this part of the encode() method (the += 4 in particular):

buf[pos++] = intToBase64[(x >> 18) & 0x3f];
buf[pos++] = intToBase64[(x >> 12) & 0x3f];
buf[pos++] = intToBase64[(x >> 6) & 0x3f];
buf[pos++] = intToBase64[x & 0x3f];
currentLinePos += 4;
if (lineLength > 0 && lineLength <= currentLinePos) {
  System.arraycopy(lineSeparator, 0, buf, pos, lineSeparator.length);
  pos += lineSeparator.length;
  currentLinePos = 0;
}

I can't think of a good name for this magic number! Maybe ENCODE_UNIT ?

[Sebb]

Base64 constructors have been made public:

URL: http://svn.apache.org/viewvc?rev=669616&view=rev
Log:
CODEC-69 (part) - make Base64 constructors public

[Sergei S. Ivanov]

Linked this issue and CODEC-8