[TIKA-354] ProfilingHandler should take a length-limiting parameter

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.5
• Component/s: languageidentifier
• Assignee: Kenneth William Krugler
• Reporter: Vivek Magotra
• Created: 16/Dec/09 01:23
• Updated: 01/Mar/15 23:02
• Resolved: 01/Mar/15 06:58

Description

ProfilingHandler currently parses the entire document (thereby analyzing n-grams for the entire doc).
ProfilingHandler should take a length-limiting parameter that allows a user to specify the amount of data that should get analyzed.
In fact, by default that limit should be set to something like 8K.