Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-4386

Variable expansion doesn't work in DIH SimplePropertiesWriter's filename

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      I'm testing Solr 4.1, but I've run into some problems with DataImportHandler's new propertyWriter tag.
      I'm trying to use variable expansion in the `filename` field when using SimplePropertiesWriter.

      Here are the relevant parts of my configuration:

      conf/solrconfig.xml
      -----------------------------------------------------------------------------
      <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
      <lst name="defaults">
      <str name="config">db-data-config.xml</str>
      </lst>

      <lst name="invariants">
      <!-- country_code is available -->
      <str name="country_code">${country_code}</str>
      <!-- In the real config, more variables are set here -->
      </lst>
      </requestHandler>

      conf/db-data-config.xml
      -----------------------------------------------------------------------------
      <dataConfig>
      <propertyWriter
      dateFormat="yyyy-MM-dd HH:mm:ss"
      type="SimplePropertiesWriter"
      directory="conf"
      filename="${dataimporter.request.country_code}.dataimport.properties"
      />

      <dataSource type="JdbcDataSource"
      driver="${dataimporter.request.db_driver}"
      url="${dataimporter.request.db_url}"
      user="${dataimporter.request.db_user}"
      password="${dataimporter.request.db_password}"
      batchSize="${dataimporter.request.db_batch_size}" />
      <document>
      <entity name="item"
      query="my normal SQL, not really relevant
      – country=${dataimporter.request.country_code}">
      <field column="id"/>
      <!-- ...more field tags... -->

      <field column="$deleteDocById"/>
      <field column="$skipDoc"/>
      </entity>
      </document>
      </dataConfig>

      If country_code is set to "gb", I want the last_index_time to be read and written in the file conf/gb.dataimport.properties, instead of the default conf/dataimport.properties

      The variable expansion works perfectly in the SQL and setup of the data source, but not in the property writer's filename field.

      When initiating an import, the log file shows:

      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter maybeReloadConfiguration
      INFO: Loading DIH Configuration: db-data-config.xml
      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
      INFO: The field :$skipDoc present in DataConfig does not have a counterpart in Solr Schema
      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
      INFO: The field :$deleteDocById present in DataConfig does not have a counterpart in Solr Schema
      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter loadDataConfig
      INFO: Data Configuration loaded successfully
      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.DataImporter doFullImport
      INFO: Starting Full Import
      Jan 30, 2013 11:25:42 AM org.apache.solr.handler.dataimport.SimplePropertiesWriter readIndexerProperties
      WARNING: Unable to read: ${dataimporter.request.country_code}.dataimport.properties

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shalin Shalin Shekhar Mangar Assign to me
            birgander Jonas Birgander
            Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment