> What's your TODO/FIXME convention?
I don't have a strong convention. Do you?
What I meant here is that creating a new encoder per datum is unacceptable. A JsonEncoder compiles the schema as a grammar and is meant to be reused. Jackson's JsonGenerator is reusable, but unfortunately inserts a space before all but the first item for some unknown reason that is at least a misfeature for our purposes. Looking at the thrift-protobuf-compare benchmarks, they do create a new JsonGenerator per datum, so they must be lightweight. But we still need to avoid re-compiling the grammer per datum.
> The core issue is that we've got two different things going on: we're both line-oriented and JSON-oriented.
You're right. I munged this together.
So we perhaps should consider lines the container, parsing them first, then parsing json within them, as your patch did. But we should not create a new Decoder per line, since it also compiles the grammar.
To address both of these, perhaps we should add methods:
static Parser JsonEncoder#parse(Schema);
static Parser JsonDecoder#parse(Schema);
Then we could create the parser once outside the loop and then re-create lightweight objects within the loop and hope that doesn't hurt performance much. My first choice would be to make encoders and decoders reusable, but that does not appear possible currently with Jackson.
> It doesn't seem that using the ValidatingDecoder makes it check that, but i could be wrong.
I believe that the Json tokens are actually strictly checked.