[SOLR-4386] Variable expansion doesn't work in DIH SimplePropertiesWriter's filename

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 4.1
• Component/s: contrib - DataImportHandler
• Labels: dataimport
• Assignee: Shalin Shekhar Mangar
• Reporter: Jonas Birgander
• Created: 30/Jan/13 15:50
• Updated: 29/Aug/20 19:44
• Resolved: 29/Aug/20 19:44

Description

I'm testing Solr 4.1, but I've run into some problems with DataImportHandler's new propertyWriter tag.
I'm trying to use variable expansion in the `filename` field when using SimplePropertiesWriter.

Here are the relevant parts of my configuration:

conf/solrconfig.xml
-----------------------------------------------------------------------------
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">db-data-config.xml</str>
</lst>

<lst name="invariants">
<!-- country_code is available -->
<str name="country_code">${country_code}</str>
<!-- In the real config, more variables are set here -->
</lst>
</requestHandler>

conf/db-data-config.xml
-----------------------------------------------------------------------------
<dataConfig>
<propertyWriter
dateFormat="yyyy-MM-dd HH:mm:ss"
type="SimplePropertiesWriter"
directory="conf"
filename="${dataimporter.request.country_code}.dataimport.properties"
/>

<dataSource type="JdbcDataSource"
driver="${dataimporter.request.db_driver}"
url="${dataimporter.request.db_url}"
user="${dataimporter.request.db_user}"
password="${dataimporter.request.db_password}"
batchSize="${dataimporter.request.db_batch_size}" />
<document>
<entity name="item"
query="my normal SQL, not really relevant
– country=${dataimporter.request.country_code}">
<field column="id"/>
<!-- ...more field tags... -->

<field column="$deleteDocById"/>
<field column="$skipDoc"/>
</entity>
</document>
</dataConfig>

If country_code is set to "gb", I want the last_index_time to be read and written in the file conf/gb.dataimport.properties, instead of the default conf/dataimport.properties

The variable expansion works perfectly in the SQL and setup of the data source, but not in the property writer's filename field.

When initiating an import, the log file shows:

Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter maybeReloadConfiguration
INFO: Loading DIH Configuration: db-data-config.xml
Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
INFO: The field :$skipDoc present in DataConfig does not have a counterpart in Solr Schema
Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
INFO: The field :$deleteDocById present in DataConfig does not have a counterpart in Solr Schema
Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter loadDataConfig
INFO: Data Configuration loaded successfully
Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter doFullImport
INFO: Starting Full Import
Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.SimplePropertiesWriter readIndexerProperties
WARNING: Unable to read: ${dataimporter.request.country_code}.dataimport.properties

Issue Links

• is superceded by: SOLR-14783 Remove DIH from 9.0