Wouldn't it be better if the payload was an additional FST output (e.g. in the case its an ID or something).
I agree packing surface form + payload into a single output is weirdish ... making it a separate FST output would be nice, but then I dreaded the different generics (sometimes FST<Pair> and other times FST<Triple> or FST<Pair<Pair>>).
I havent fully thought about the benefits, but i guess an ID value as a separate output could be used for
other things, like participating in the top-N comparator to simplify the tie-break logic (which if i recall,
is super-hairy for this suggester).
True! Though, we could do the same thing w/ the current approach ... eg break out the payload and let the app check it for accept / reject (somewhere there is a patch to expose accept method in AnalyzingSuggester...).
It might also be more efficient as a separate output, but I havent thought about all the cases there either.
its like if payloads dont exist we could just use the empty string or whatever and there is really not much
Yeah it could be. Hmm, every arc would add a byte (mostly, sometimes more I guess if payload is longish), but if the payloads can be shared well, then the overall FST could be smaller ...
I agree we should explore it!