Major Description
Analytics engines like Hive etc cannot handle recursive schemas: schemas where inner fields can refer to the wrapping type.
This Jira proposes that we provide support for automatic recursion removal in data during data ingestion.
The simple proposal is to just drop the fields in the schema that introduce the recursion.
e.g. (pseudo-schema)
User
{ string name; User friend; }gets converted to :
User
{ string name; }
A more sophisticated solution would be to do one or two levels of "schema-unrolling" before dropping data.
e.g.
output schema with one-level unrolling would look like:
User
{ string name; User1 friend; }User 1
{ string name; }
Attachments
Issue Links
- links to
