[JOSHUA-128] Lattice issues

• Type: Bug
• Status: Reopened
• Resolution: Unresolved
• Fix Version/s: 6.2
• Assignee: Matt Post
• Reporter: Matt Post
• Created: 26/Jul/13 01:45
• Updated: 20/Jun/16 18:29

Description

Joshua truncates sentences to 200 tokens, but turns them into a lattice first and computes the shortest path between all pairs. This causes huge problems for buggy input (e.g., https://groups.google.com/forum/#!topic/joshua_support/-2PykKqbnU4 ). A couple things that should change:

• Don't apply an O(n^2) algorithm to buggy input. Maybe only call calculateAllPairsShortestPaths() lazily.

• Compute it more efficiently (for example, if you know the input is just a sentence, you could add some shortcuts)