SolrSharp - a C# client API for Solr

• Key: SOLR-205
• Type: New Feature
• Status: Reopened
• Priority: Minor
• Assignee: Jeff Rodenburg
• Reporter: Jeff Rodenburg
• Component/s: clients - C#

Description

SolrSharp is a client API written in C# using the .Net framework to interact with Apache Solr.

Comments

[Jeff Rodenburg]

This library and project consists of a few different chunks:

/src - the source code for the library.
/docs/api - MSDN-style documentation for the library (the MS world equivalent of javadoc).
/example - working examples for the library.

The namespace for the source code is "org.apache.solr.SolrSharp"

Are there any documentation requirements for Apache compliance?

One administrative note: I have a signed agreement on file with the Apache Foundation for code contributions, indemnification of my employer, etc. This is the standard doc that's required for all committers (I have committer status on the Lucene.Net project). I assume this applies for any Apache project.

[Jeff Rodenburg]

Full complement of files for distribution. Here's what's included:

/src
Source code for the SolrSharp project
/docs
MSDN-style web documentation of the SolrSharp library
/example
An example application that implements the library. Uses the sample configuration from Solr.

More of this is forthcoming, but this is an initial release to get it in the hands of users.

[Otis Gospodnetic]

Jeff & Co:
What is the status of this as far as getting it into the repository?
Something to do right after 1.2 is fully released?

It's still in sync with 1.2 codebase and works, right Jeff?

[ms]

I am having difficulty using SolrSharp in a VB.NET project. I am using .NET 2.0 uncer Windows Vista (VS 2005). The problem is that the document does not pass "IsValidUpdateIndexDocument'
--------------------------------------------------------------
I create a SolrSharp document thus:

Imports System.Xml.Serialization
Imports org.apache.solr.SolrSharp.Indexing

<Serializable(), XmlRoot("add")> _
Public Class SolrDocument

Inherits UpdateIndexDocument

Public Sub New()
End Sub

Public Sub New(ByVal id As String, ByVal Diagnosis As String)
Me.Add(New IndexFieldValue("id", id))
Me.Add(New IndexFieldValue("Diagnosis", Diagnosis))
End Sub
End Class

Then I try to post it like this:

Dim solrSearcher As SolrSharp.Configuration.SolrSearcher _
= SolrSharp.Configuration.SolrSearchers.GetSearcher(SolrSharp.Configuration.Mode.Read)

Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher)
Dim doc As New SolrDocument(txtID.Text, txtDiagnosis.Text)

If Not solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc) Then MessageBox.Show("Cannot validate document: ")

Try

SolrU.PostToIndex(doc, True)

Catch ex As Exception
MessageBox.Show(ex.ToString)
End Try

[Jeff Rodenburg]

I'm not seeing comments that I've added via email response displayed on this issue, so I'll add them manually.

Regarding question from Otis G. about repository access and 1.2 compliance:

• I can't confirm right now (other project keeping me from this at the present.) On evaluation it doesn't appear that there are any breaking changes at this point, other than Lucene query syntax requiring a change for field values with semi-colons. However, there are new features in both this release as well as 1.1 that have not been incorporated into the library (defaults used as opposed to allowing choices where those exist.) I will update JIRA with a confirmation indicating the library is 1.2 compliant.
• Repository access has been requested.

[Jeff Rodenburg]

Regarding ms IsValidUpdateIndexDocument issue:

IsValidUpdateIndexDocument compares the fields of the referenced IndexDocument to the fields as read from the Solr configuration for the Solr instance you're updating. If the (non-copied) fields that are read from the solrconfig.xml file are not present in the list of fields from the IndexDocument, IsValidUpdateIndexDocument will fail.

In your example, you should have only two fields defined in solrconfig.xml of the solr instance you're updating.

[Jeff Rodenburg]

Solrsharp has been validated against the Solr 1.2 release. Validation was made using the example application that's available with the Solrsharp code.

• The code has been moved to a new location and is now accessible via subversion. Many thanks to Ryan McKinley for hosting the codebase. You can find it at:

http://solrstuff.org/svn/solrsharp

• A new folder has been added: docs/api. We have MSDN-style documentation to help explain the full library. When you update from the repository, just point your browser to the local file at /docs/api/index.html.

[ms]

Jeff,
Thanks. I have it working against Solr 1.2. I am trying to figure out how to delete a document from the index. This is the code I am using and it is not working. TIA

Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher)
Dim doc As New org.apache.solr.SolrSharp.Indexing.DeleteIndexDocument
doc.Id = "a"
SolrU.PostToIndex(doc, True)
SolrU.Optimize()

[Jeff Rodenburg]

There was a bug in DeleteIndexDocument, which has been updated. Deleting by query vs. unique key was not being handled properly. The source at http://solrstuff.org/svn/solrsharp has been updated (documentation to updated later.)

ms - for your scenario, this issues a delete-by-id request using the xml syntax "<delete><id>a</id></delete>". This is not the same as a delete-by-query, if that's the intention. Please look at the example for code that shows how to delete by query.

[ms]

Jeff- I will check the code at the link you provided. However, looking at the Examples, I do not see code sample for delete by query. My intention is to delte all documents/range of documents prior to (partially) reb-uilding the index.

[Jeff Rodenburg]

The Example application includes some code for delete by query, but it is commented out. The particular code looks like:

//List<QueryParameter> listQP = new List<QueryParameter>();
//listQP.Add(new QueryParameter("id", "101"));
//QueryParameterCollection queryParameterCollection = new QueryParameterCollection("delete", listQP);
//Query query = new Query();
//query.AddQueryParameters(queryParameterCollection, ParameterJoin.AND);
//DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument(query);

DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument("101");
oUpdate.PostToIndex(deleteIndexDocument, true);

Uncomment those and comment out the single DeleteIndexDocument("101") line to execute a delete-by-query.

[Jeff Rodenburg]

Current Solrsharp library, compliant with Solr version 1.2. For the latest code, please see the Subversion repository at http://solrstuff.org/svn/solrsharp

[ms]

I am using solrsharp-1.2-07082007 - I have a dynamicField declared in my schema.xml file as
<dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
-but, if I try to add a field using my vb.net application
doc.Add("id_demo", s)
where is a string value, the document does fails
solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc)
MS

[Jeff Rodenburg]

Includes updates to SOLR-304. Replaces solrsharp-1.2-07082007.zip

[ms]

With solrsharp-1.2-07082007 I am able to use dynamicFields. However, I have a problem with copyFiels that reference dynamicFields.
Consider the above example of
<dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
if I now have
<copyField source="id_demo" dest="demographics" />
then I must also have
<field name="demographics" type="text_ws" indexed="true" stored="false"/>
Now, if I use Solr#, I have to have
doc.Add("demographics", "")
or it will fail IsValidUpdateIndexDocument. But this is causing the index to be corrupted. Also, this defeats the purpose of copyFields - which is a server side mechanism to aggregate fields.
Could we have solr# raise IsValidUpdateIndexDocument = false only if the field has "Isrequired = true"? tia

[ms]

Another possible bug - not sure if this is a bug with Solr# or Solr. To reproduce, create a dynamicField and copyField in schema like this

<dynamicField name="*_demo" type="string" indexed="true" stored="true"/>
<copyField source="*_demo" dest="demographics_demo" />

Now, add TWO fields to match your dynamic field using solr# - for example:

doc.Add("id_demo", "lorem")
doc.Add("sample_demo", "ipsum")

When I add this document to a new Index, I get a corrupted index.

[Jeff Rodenburg]

> Consider the above example of
<dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
> if I now have
<copyField source="id_demo" dest="demographics" />
> then I must also have
<field name="demographics" type="text_ws" indexed="true" stored="false"/>

Let me make sure I understand the scenario here:

• dynamic field = "*_demo"
• copy field source = "id_demo" (as captured by dynamic field), with dest = "demographics"

I'll test this scenario myself to see what occurs.

[ms]

Jeff
Please let me know if you were able to get this to work.
The problem I have is that the the client using Solr# needs to supply a value to the declared field "demographics" - or ti will fail IsValidUpdateIndexDocument.
A related problem that I am having is that if you add 2 or more dynamic fields ("id_demo" and "somethingelse_demo") - there is no error message - but, using Luke I see that the document has not been added to the index. TIA

[Jeff Rodenburg]

Apologies for the delay in the response, occupied with starting a new venture.

First, IsValidUpdateIndexDocument is not a required call to update a solr index. It is intended to be a runtime check for a document's structure, possibly more useful when added an nunit test routine than being called in production. Nonetheless, it should still be applicable in this scenario.

On testing with the dynamic field scenario, I've not been able to reproduce the issue. I'm continuing with that to see if I can determine if there's a scenario that solrsharp should be handling vs. an issue with solr.

[Jeff Rodenburg]

Added support for highlighting, as outlined in https://issues.apache.org/jira/browse/SOLR-338

[Jeff Rodenburg]

Latest release that includes updates for highlighting and debug parameter options

[Tomer Gabel]

The following small patch (generated with diff, sorry about the format if not appropriate) has the following features:

• Switch to a (IMO...) tidier type mapping system with a static dictionary instead of an if statement hierarchy
• Adds missing type mappings (long, unsigned types etc.) – note the warning about small (less than 32 bit) data types!
• Adds constructor overloads for abstract QueryBuilder that accept SolrSearcher instead of reading from configuration (we prefer to integrate with our own configuration provider)

(Edited to include attachment instead of text block)

[Tomer Gabel]

"diff" output for the above changes

[Jeff Rodenburg]

Great patch, thanks Tomer. I just looked through the patch, here are a few comments:

• SolrType.cs/TypeExpression(...): replace the if/else lookup scheme for type mappings with a static type Dictionary for lookups.
  The typemap lookup is much cleaner than the if/else sequence. Shorter code, of which I'm a fan. Nice optimization, thanks.

• QueryBuilder.cs: add public constructors that take a SolrSearcher instance. There are 6 or 7 constructors, and they would all be extended with an additional SolrSearcher instance.

I'm unsure about the best way to go about freely associating SolrSearcher instances with QueryBuilder instances, outside of configuration control. QueryBuilder implies searchability against a solr server, while the given client configuration may make that server update-only (the SearcherMode property on SolrSearcher.) The idea behind the configuration settings for servers (Read/Write settings) is to ensure updates and queries occur in the right places. At a minimum, there should be a check on the SolrSearcher instance passed to a QueryBuilder constructor that ensures the SearcherMode supports Read.

Another matter is encapsulation around the SolrSearcher instance and its query path. Referring to http://wiki.apache.org/solr/SolrRequestHandler, the "select/" path for queries will technically work but also can short-circuit defined handlers already defined on a solr server. The SolrSearcher instance should respect the server configuration on the solr instance. This is not currently in place in the solrsharp code, but is something that we should add. This involves wiring up Request Handlers, which haven't been done yet.

My thoughts on this patch:
1) Apply the change to SolrType.TypeExpression(...)
2) Apply the constructor changes to QueryBuilder, but modify them to incorporate checking on SearcherMode properties in the constructor.
3) Begin evaluation of strongly-typed Request Handlers to control the SolrSearcher Url property.