[LUCENE-6605] Set payloads in token within TokenGroup in Highlighter

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.2.1
• Labels: highlighter
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: robert.onslow
• Created: 25/Jun/15 08:34
• Updated: 28/Aug/22 14:36

Description

I have a tokenizer which sets the payload attribute and which drives the highlighter. In Formatter.highlightTerm() I need to examine that payload on the token. However TokenGroup.addToken() doesn't currently set the payload on the token which is created and fed through to Formatter.highlightTerm() within the TokenGroup.

I would suggest, TokenGroup.addToken():

token.setPayload(payloadAtt.getPayload());

having of course registered the payload attribute during initialisation of TokenGroup with:

payloadAtt = tokenStream.addAttribute(PayloadAttribute.class);