Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
6.6
-
None
-
None
-
Elasticsearch 5.5.2, Build: b2f0c09/2017-08-14T12:33:14.154Z, JVM: 1.8.0_144
JVM java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)OS Linux 3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
Elasticsearch 5.5.2, Build: b2f0c09/2017-08-14T12:33:14.154Z, JVM: 1.8.0_144 JVM java version "1.8.0_144" Java(TM) SE Runtime Environment (build 1.8.0_144-b01) Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode) OS Linux 3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
-
New
Description
I noticed that the uax_url_email tokenizer splits urls in multiple tokens in presence of digits, ".", "-"
I opened a issue on elasticsearch github repo (https://github.com/elastic/elasticsearch/issues/27309) because I noticed this strange behaviour.
Their answer was
The uax_url_email tokenizer tokenizes URLs and email addresses, but in order to recognize a token as a URL it must include the scheme (usually HTTP:// or https://):
Additionally, this tokenizer belongs to Lucene. Could you open this issue at https://lucene.apache.org/core/ instead?
URLs are defined by RFC1738 and extended by RFC1808.
In RFC1808 Relative URLs are explained, and this allows scheme-less URLs.
I would expect uax_url_email to tokenize correctly also scheme-less and relative URL.