Add support for application-specific typecodes to typed bytes

• Key: HADOOP-5450
• Type: Improvement
• Status: Resolved
• Resolution: Fixed
• Priority: Blocker
• Assignee: Klaas Bosteels
• Reporter: Klaas Bosteels
• Fix Version/s: 0.21.0

Description

For serializing objects of types that are not supported by typed bytes serialization, applications might want to use a custom serialization format. Right now, typecode 0 has to be used for the bytes resulting from this custom serialization, which could lead to problems when deserializing the objects because the application cannot know if a byte sequence following typecode 0 is a customly serialized object or just a raw sequence of bytes. Therefore, a range of typecodes that are treated as aliases for 0 should be added, such that different typecodes can be used for application-specific purposes.

Comments

[Klaas Bosteels]

The attached patch makes sure that the typecode range 50-200 can be used for application-specific purposes. As an example, it also makes TypedBytesWritableInput and TypedBytesWritableOutput use typecode 50 for serializing Writables that do not match with any of the types supported by the typed bytes format.

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12401846/HADOOP-5450.patch
against trunk revision 752292.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The patch failed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/46/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/46/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/46/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/46/console

This message is automatically generated.

[Klaas Bosteels]

The failed unit tests are not related to the patch.

[Klaas Bosteels]

This should really go into 0.21, so making it a blocker for that release.

[Owen O'Malley]

I just committed this. Thanks, Klaas!

[Hudson]

Integrated in Hadoop-trunk #797 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/797/)
. Revert accidental inclusion.
. Add application-specific data types to streaming's typed bytes
interface. (Klaas Bosteels via omalley)