You're right, I've left this ticket untouched while working through the initial subtasks.
The order-preserving serialization is a critical component of the work. I think this is a feature that HBase absolutely must provide if there's to be any hope for interoperability. I also think the serialization format is necessary but not sufficient. An HBase that ships with an API for describing data types and implementations of a set of common definitions takes the next step in interoperability. By defining the type interface, type implementations provided by 3rd parties become pluggable – it becomes feasible for a user to plug a type from Phoenix into their Kiji application. Systems like Phoenix, Kiji, and HCatalog are all choices for defining and managing schema. It may be the case that HBase should define the schema interfaces as well, but that's definitely beyond the scope here. But if those tools are going to interoperate, they need a common language of types with which to do so. Serialization, IMHO, is insufficient.
I don't know if there's a new project to be built out of this work. I see no need to create such a thing when the needs and use are not yet proven. The introduction of types in HBase will shake things up enough as it is, let's see how people and projects use them before promoting this stuff to its own project.
Yes, the serialization formats defined in
HBASE-8201 are designed to be language agnostic. It's highly likely that I've missed some critical details here or there in the specification. Time will tell