[LUCENE-10013] Document contains at least one immense term in field (whose UTF8 encoding is longer than the max length 32766)

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Done
• Affects Version/s: 4.10.4
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Arvind Kumar Sahu
• Created: 22/Jun/21 14:08
• Updated: 28/Aug/22 16:23
• Resolved: 25/Jun/21 11:04

Description

Hi Team,

Currently we are using Lucene 4.10.4 version. We are getting the below error:

"Document contains at least one immense term in field (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[-41, -103, -41, -87, -41, -103, -41, -111, -41, -108, 32, 56, 56, 45, -41, -108, -41, -111, -41, -107, -41, -89, -41, -88, 44, 32, 40, 32, 49, 51]...', original message: bytes can be at most 32766 in length; got 35169".

We understand from the Lucene JIRA ticket LUCENE-5472 Long terms should generate a RuntimeException, not just infoStream - ASF JIRA (apache.org), this issue has been resolved in 4.8 and 6.0.

Please confirm us if this fix is included in 4.10.4.