[SOLR-1916] investigate DIH use of default locale

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1, 4.0-ALPHA
• Fix Version/s: 4.1, 6.0
• Component/s: contrib - DataImportHandler
• Assignee: James Dyer
• Reporter: Robert Muir
• Created: 17/May/10 20:20
• Updated: 09/May/16 18:46
• Resolved: 20/Nov/12 19:56

Description

This is a spinoff from LUCENE-2466.

In this issue I changed my locale to various locales and found some problems in Lucene/Solr triggered by use of the default Locale.

I noticed some use of the default-locale for Date operations in DIH (TimeZone.getDefault/Locale.getDefault) and, while no tests fail, I think it might be better to support a locale parameter for this.

The wiki documents that numeric parsing can support localized numerics formats: http://wiki.apache.org/solr/DataImportHandler#NumberFormatTransformer

In both cases, I don't think we should ever use the default Locale. If no Locale is provided, I find that new Locale("") <-- Unicode Root Locale, is a better default for a server situation in a lot of cases, as it won't change depending on the computer, or perhaps we just make Locale params mandatory for this.

Finally, in both cases, if localized numbers/dates are explicitly supported, I think we should come up with a test strategy to ensure everything is working. One idea is to do something similar to or make use of Lucene's LocalizedTestCase.

Issue Links

• incorporates: SOLR-4095 DIH - NumberFormatTransformer & DateFormatTransformer should default to the Root Locale
• incorporates: SOLR-4096 DIH - FileDataSource & FieldReaderDataSource should default to UTF-8 charset
• incorporates: SOLR-4086 Refactor DIH - VariableResolver & Evaluator
• is related to: SOLR-4051 DIH Delta updates do not work for all locales
• relates to: SOLR-3365 Data import using local time to mark last_index_time
• relates to: SOLR-2201 DIH - Support specific timezones with formatDate function