[NUTCH-2957] indexer-solr / Solr schema: add fall-back field definitions for unknown index fields

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.18
• Fix Version/s: 1.19
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 06/Aug/22 13:25
• Updated: 13/Mar/24 14:51
• Resolved: 17/Aug/22 13:43

Description

If a Nutch index field isn't defined in the schema.xml (shipped with indexer-solr) the indexer fails, e.g. here by passing a static index field:

$> bin/nutch indexchecker \
     -Dplugin.includes='protocol-http|parse-html|index-static|indexer-solr' \
     -D'index.static=foo:bar' \
     -doIndex http://localhost/
Exception in thread "main" org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr/nutch: copyField dest :'foo_str' is not an explicit field and doesn't match a dynamicField.
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290)
        at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:250)
        ...

Would be nice to have a fall-back field definition which allows for unspecified fields.

Issue Links

• fixes: NUTCH-2945 Solr Index Writer pluging schema.xml missing a copyToField
• links to: GitHub Pull Request #739