[LUCENE-9914] Modernize Emoji regeneration scripts

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 9.0
• Lucene Fields: New
• Assignee: Dawid Weiss
• Reporter: Dawid Weiss
• Created: 07/Apr/21 19:03
• Updated: 28/Aug/22 16:20
• Resolved: 12/Apr/21 18:17

Description

These are perl scripts... I don't think they had ant tasks in 8x and they haven't been used in a while. They don't seem too scary (for perl) - just fetch emoji unicode descriptions and parse them into a jflex macro and a test case.

It'd be good to convert them to use python, groovy or even java so that they fit better in the build system. Alternatively - perhaps there is a way to get these codepoint properties from Java directly?

Issue Links

• is related to: LUCENE-9871 Achieve build system perfection (with progress)
• relates to: LUCENE-9655 Add code regeneration for generateEmojiTokenizationTest.pl
• relates to: LUCENE-9656 Add code regeneration for generateJavaUnicodeWordBreakTest.pl
• links to: GitHub Pull Request #78