[LUCENE-4927] Prevent underflow in NB classifier likelihood calculation

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.2
• Fix Version/s: 4.7, 6.0
• Component/s: modules/classification
• Lucene Fields: New
• Assignee: Tommaso Teofili
• Reporter: Tommaso Teofili
• Created: 11/Apr/13 13:58
• Updated: 28/Aug/22 13:44
• Resolved: 25/Nov/13 08:10

Description

Current likelihood calculation multiplies probabilities (whose values are between 0 and 1) thus having longish docs with unfrequent words for some class/category may lead to multiple double multiplications to return 0 even if that's not the correct value (thus assigning such a class 0 probability too).

Probably using loglikelihood and/or BigDecimals may help.