[TIKA-3145] Add a content digester to tika-eval text stats

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.25
• Assignee: Tim Allison
• Reporter: Tim Allison
• Created: 23/Jul/20 21:34
• Updated: 24/Jul/20 22:43
• Resolved: 24/Jul/20 21:20

Description

When comparing files, it can be useful to digest the text contents so that users can identify files that may have duplicate content but different overall digests. Let's add a content digester to tika-eval's text stats calculator.

See: https://builds.apache.org/job/nutch-trunk/javadoc/org/apache/nutch/crawl/TextMD5Signature.html