[LUCENE-34] e-mail token in StandardTokenizer.jj does not match valid e-mail addresses

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: modules/analysis

Environment

Operating System: Linux
Platform: PC

• Bugzilla Id: 9015
• Assignee: Lucene Developers
• Reporter: Dale Anson
• Created: 12/May/02 23:56
• Updated: 28/Aug/22 11:10
• Resolved: 27/May/06 01:35

Description

E-mail token in StandardTokenizer.jj does not match many valid e-mail
addresses. See line 106:

<EMAIL: <ALPHANUM> "@" <ALPHANUM> ("." <ALPHANUM>)+ >

For example, neither [e-mail] (because of the dash) nor
[e-mail] (because of the first dot and the dash) match.
the following is slightly better, but does not come close to meeting the
specifications of RFC 822:

<EMAIL: <ALPHANUM> ("."|"" <ALPHANUM>)+ "@" <ALPHANUM> ("."|"" <ALPHANUM>)+
>

This is being reported against the May 11 nightly build (I compiled from
source using the supplied Ant build file on RedHat Linux 7.2, jikes, javacc
2.0, and Sun Linux JDK 1.4), however, I originally ran across this problem in
Lucene 1.2 rc4.