Details
-
Bug
-
Status: Reopened
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Incomplete n-grams have no estimate, which creates a problem when comparing small phrases (early in decoding) and for short sentences. To fix this, the LanguageModelFF should compute partial n-gram scores, caching the result and the n-gram length in the DP state so that it can be subtracted off later.
Another advantage of this is that it would likely allow a large simplification of the FeatureFunction interface, since we would no longer need a computeFinal distinction