[SOLR-11250] Add new LTR model which loads the model definition from the external resource

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 7.2, 8.0
• Component/s: contrib - LTR
• Assignee: Christine Poerschke
• Reporter: Yuki Yano
• Created: 17/Aug/17 06:02
• Updated: 11/Dec/19 19:06
• Resolved: 01/Dec/17 15:46

Description

example of committed change's usage:

{
  "class" : "org.apache.solr.ltr.model.DefaultWrapperModel",
  "name" : "myWrapperModelName",
  "params" : {
    "resource" : "models/myModel.json"
  }
}

---

original summary:
We add new model which contains only the location of the external model and loads it during the initialization.
By this procedure, large models which are difficult to upload to ZooKeeper can be available.

The new model works as the wrapper of existing models, and deligates APIs to them.
We add two classes by this patch:

• ExternalModel : a base class for models with external resources.
• URIExternalModel : an implementation of ExternalModel which loads the external model from specified URI (ex. file:, http:, etc.).

For example, if you have a model on the local disk "file:///var/models/myModel.json", the definition of URIExternalModel will be like the following.

{
  "class" : "org.apache.solr.ltr.model.URIExternalModel",
  "name" : "myURIExternalModel",
  "features" : [],
  "params" : {
    "uri" : "file:///var/models/myModel.json"
  }
}

If you use LTR with model=myURIExternalModel, the model of myModel.json will be used for scoring documents.

Issue Links

• is related to: SOLR-11049 Solr in cloud mode silently fails uploading a big LTR model
• is related to: SOLR-4793 Solr Cloud can't upload large config files ( > 1MB) to Zookeeper
• is related to: SOLR-12590 Improve Solr resource loader coverage in the ref guide
• relates to: SOLR-11838 explore supporting Deeplearning4j NeuralNetwork models
• relates to: SOLR-11852 retrieval of contrib/ltr model definitions via a streaming expression
• relates to: SOLR-14057 simplify a <lib/> directive in LTR ref guide

Sub-Tasks

• make LTRScoringModel model more extensible: Closed