Solr
  1. Solr
  2. SOLR-205

SolrSharp - a C# client API for Solr

    Details

    • Type: New Feature New Feature
    • Status: Reopened
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: clients - C#
    • Labels:
      None
    • Environment:

      Microsoft Windows, .Net Framework 2.0

      Description

      SolrSharp is a client API written in C# using the .Net framework to interact with Apache Solr.

      1. solrsharp-1.2-08302007.zip
        1.30 MB
        Jeff Rodenburg
      2. patch
        10 kB
        Tomer Gabel

        Activity

        Hide
        Jeff Rodenburg added a comment -

        Great patch, thanks Tomer. I just looked through the patch, here are a few comments:

        • SolrType.cs/TypeExpression(...): replace the if/else lookup scheme for type mappings with a static type Dictionary for lookups.
          The typemap lookup is much cleaner than the if/else sequence. Shorter code, of which I'm a fan. Nice optimization, thanks.
        • QueryBuilder.cs: add public constructors that take a SolrSearcher instance. There are 6 or 7 constructors, and they would all be extended with an additional SolrSearcher instance.

        I'm unsure about the best way to go about freely associating SolrSearcher instances with QueryBuilder instances, outside of configuration control. QueryBuilder implies searchability against a solr server, while the given client configuration may make that server update-only (the SearcherMode property on SolrSearcher.) The idea behind the configuration settings for servers (Read/Write settings) is to ensure updates and queries occur in the right places. At a minimum, there should be a check on the SolrSearcher instance passed to a QueryBuilder constructor that ensures the SearcherMode supports Read.

        Another matter is encapsulation around the SolrSearcher instance and its query path. Referring to http://wiki.apache.org/solr/SolrRequestHandler, the "select/" path for queries will technically work but also can short-circuit defined handlers already defined on a solr server. The SolrSearcher instance should respect the server configuration on the solr instance. This is not currently in place in the solrsharp code, but is something that we should add. This involves wiring up Request Handlers, which haven't been done yet.

        My thoughts on this patch:
        1) Apply the change to SolrType.TypeExpression(...)
        2) Apply the constructor changes to QueryBuilder, but modify them to incorporate checking on SearcherMode properties in the constructor.
        3) Begin evaluation of strongly-typed Request Handlers to control the SolrSearcher Url property.

        Show
        Jeff Rodenburg added a comment - Great patch, thanks Tomer. I just looked through the patch, here are a few comments: SolrType.cs/TypeExpression(...): replace the if/else lookup scheme for type mappings with a static type Dictionary for lookups. The typemap lookup is much cleaner than the if/else sequence. Shorter code, of which I'm a fan. Nice optimization, thanks. QueryBuilder.cs: add public constructors that take a SolrSearcher instance. There are 6 or 7 constructors, and they would all be extended with an additional SolrSearcher instance. I'm unsure about the best way to go about freely associating SolrSearcher instances with QueryBuilder instances, outside of configuration control. QueryBuilder implies searchability against a solr server, while the given client configuration may make that server update-only (the SearcherMode property on SolrSearcher.) The idea behind the configuration settings for servers (Read/Write settings) is to ensure updates and queries occur in the right places. At a minimum, there should be a check on the SolrSearcher instance passed to a QueryBuilder constructor that ensures the SearcherMode supports Read. Another matter is encapsulation around the SolrSearcher instance and its query path. Referring to http://wiki.apache.org/solr/SolrRequestHandler , the "select/" path for queries will technically work but also can short-circuit defined handlers already defined on a solr server. The SolrSearcher instance should respect the server configuration on the solr instance. This is not currently in place in the solrsharp code, but is something that we should add. This involves wiring up Request Handlers, which haven't been done yet. My thoughts on this patch: 1) Apply the change to SolrType.TypeExpression(...) 2) Apply the constructor changes to QueryBuilder, but modify them to incorporate checking on SearcherMode properties in the constructor. 3) Begin evaluation of strongly-typed Request Handlers to control the SolrSearcher Url property.
        Hide
        Tomer Gabel added a comment -

        "diff" output for the above changes

        Show
        Tomer Gabel added a comment - "diff" output for the above changes
        Hide
        Tomer Gabel added a comment - - edited

        The following small patch (generated with diff, sorry about the format if not appropriate) has the following features:

        • Switch to a (IMO...) tidier type mapping system with a static dictionary instead of an if statement hierarchy
        • Adds missing type mappings (long, unsigned types etc.) – note the warning about small (less than 32 bit) data types!
        • Adds constructor overloads for abstract QueryBuilder that accept SolrSearcher instead of reading from configuration (we prefer to integrate with our own configuration provider)

        (Edited to include attachment instead of text block)

        Show
        Tomer Gabel added a comment - - edited The following small patch (generated with diff, sorry about the format if not appropriate) has the following features: Switch to a (IMO...) tidier type mapping system with a static dictionary instead of an if statement hierarchy Adds missing type mappings (long, unsigned types etc.) – note the warning about small (less than 32 bit) data types! Adds constructor overloads for abstract QueryBuilder that accept SolrSearcher instead of reading from configuration (we prefer to integrate with our own configuration provider) (Edited to include attachment instead of text block)
        Hide
        Jeff Rodenburg added a comment -

        Latest release that includes updates for highlighting and debug parameter options

        Show
        Jeff Rodenburg added a comment - Latest release that includes updates for highlighting and debug parameter options
        Hide
        Jeff Rodenburg added a comment -

        Added support for highlighting, as outlined in https://issues.apache.org/jira/browse/SOLR-338

        Show
        Jeff Rodenburg added a comment - Added support for highlighting, as outlined in https://issues.apache.org/jira/browse/SOLR-338
        Hide
        Jeff Rodenburg added a comment -

        Apologies for the delay in the response, occupied with starting a new venture.

        First, IsValidUpdateIndexDocument is not a required call to update a solr index. It is intended to be a runtime check for a document's structure, possibly more useful when added an nunit test routine than being called in production. Nonetheless, it should still be applicable in this scenario.

        On testing with the dynamic field scenario, I've not been able to reproduce the issue. I'm continuing with that to see if I can determine if there's a scenario that solrsharp should be handling vs. an issue with solr.

        Show
        Jeff Rodenburg added a comment - Apologies for the delay in the response, occupied with starting a new venture. First, IsValidUpdateIndexDocument is not a required call to update a solr index. It is intended to be a runtime check for a document's structure, possibly more useful when added an nunit test routine than being called in production. Nonetheless, it should still be applicable in this scenario. On testing with the dynamic field scenario, I've not been able to reproduce the issue. I'm continuing with that to see if I can determine if there's a scenario that solrsharp should be handling vs. an issue with solr.
        Hide
        ms added a comment -

        Jeff
        Please let me know if you were able to get this to work.
        The problem I have is that the the client using Solr# needs to supply a value to the declared field "demographics" - or ti will fail IsValidUpdateIndexDocument.
        A related problem that I am having is that if you add 2 or more dynamic fields ("id_demo" and "somethingelse_demo") - there is no error message - but, using Luke I see that the document has not been added to the index. TIA

        Show
        ms added a comment - Jeff Please let me know if you were able to get this to work. The problem I have is that the the client using Solr# needs to supply a value to the declared field "demographics" - or ti will fail IsValidUpdateIndexDocument. A related problem that I am having is that if you add 2 or more dynamic fields ("id_demo" and "somethingelse_demo") - there is no error message - but, using Luke I see that the document has not been added to the index. TIA
        Hide
        Jeff Rodenburg added a comment -

        > Consider the above example of
        <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
        > if I now have
        <copyField source="id_demo" dest="demographics" />
        > then I must also have
        <field name="demographics" type="text_ws" indexed="true" stored="false"/>

        Let me make sure I understand the scenario here:

        • dynamic field = "*_demo"
        • copy field source = "id_demo" (as captured by dynamic field), with dest = "demographics"

        I'll test this scenario myself to see what occurs.

        Show
        Jeff Rodenburg added a comment - > Consider the above example of <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/> > if I now have <copyField source="id_demo" dest="demographics" /> > then I must also have <field name="demographics" type="text_ws" indexed="true" stored="false"/> Let me make sure I understand the scenario here: dynamic field = "*_demo" copy field source = "id_demo" (as captured by dynamic field), with dest = "demographics" I'll test this scenario myself to see what occurs.
        Hide
        ms added a comment -

        Another possible bug - not sure if this is a bug with Solr# or Solr. To reproduce, create a dynamicField and copyField in schema like this

        <dynamicField name="*_demo" type="string" indexed="true" stored="true"/>
        <copyField source="*_demo" dest="demographics_demo" />

        Now, add TWO fields to match your dynamic field using solr# - for example:

        doc.Add("id_demo", "lorem")
        doc.Add("sample_demo", "ipsum")

        When I add this document to a new Index, I get a corrupted index.

        Show
        ms added a comment - Another possible bug - not sure if this is a bug with Solr# or Solr. To reproduce, create a dynamicField and copyField in schema like this <dynamicField name="*_demo" type="string" indexed="true" stored="true"/> <copyField source="*_demo" dest="demographics_demo" /> Now, add TWO fields to match your dynamic field using solr# - for example: doc.Add("id_demo", "lorem") doc.Add("sample_demo", "ipsum") When I add this document to a new Index, I get a corrupted index.
        Hide
        ms added a comment -

        With solrsharp-1.2-07082007 I am able to use dynamicFields. However, I have a problem with copyFiels that reference dynamicFields.
        Consider the above example of
        <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
        if I now have
        <copyField source="id_demo" dest="demographics" />
        then I must also have
        <field name="demographics" type="text_ws" indexed="true" stored="false"/>
        Now, if I use Solr#, I have to have
        doc.Add("demographics", "")
        or it will fail IsValidUpdateIndexDocument. But this is causing the index to be corrupted. Also, this defeats the purpose of copyFields - which is a server side mechanism to aggregate fields.
        Could we have solr# raise IsValidUpdateIndexDocument = false only if the field has "Isrequired = true"? tia

        Show
        ms added a comment - With solrsharp-1.2-07082007 I am able to use dynamicFields. However, I have a problem with copyFiels that reference dynamicFields. Consider the above example of <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/> if I now have <copyField source="id_demo" dest="demographics" /> then I must also have <field name="demographics" type="text_ws" indexed="true" stored="false"/> Now, if I use Solr#, I have to have doc.Add("demographics", "") or it will fail IsValidUpdateIndexDocument. But this is causing the index to be corrupted. Also, this defeats the purpose of copyFields - which is a server side mechanism to aggregate fields. Could we have solr# raise IsValidUpdateIndexDocument = false only if the field has "Isrequired = true"? tia
        Hide
        Jeff Rodenburg added a comment -

        Includes updates to SOLR-304. Replaces solrsharp-1.2-07082007.zip

        Show
        Jeff Rodenburg added a comment - Includes updates to SOLR-304 . Replaces solrsharp-1.2-07082007.zip
        Hide
        ms added a comment -

        I am using solrsharp-1.2-07082007 - I have a dynamicField declared in my schema.xml file as
        <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/>
        -but, if I try to add a field using my vb.net application
        doc.Add("id_demo", s)
        where is a string value, the document does fails
        solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc)
        MS

        Show
        ms added a comment - I am using solrsharp-1.2-07082007 - I have a dynamicField declared in my schema.xml file as <dynamicField name="*_demo" type="text_ws" indexed="true" stored="true"/> -but, if I try to add a field using my vb.net application doc.Add("id_demo", s) where is a string value, the document does fails solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc) MS
        Hide
        Jeff Rodenburg added a comment -

        Current Solrsharp library, compliant with Solr version 1.2. For the latest code, please see the Subversion repository at http://solrstuff.org/svn/solrsharp

        Show
        Jeff Rodenburg added a comment - Current Solrsharp library, compliant with Solr version 1.2. For the latest code, please see the Subversion repository at http://solrstuff.org/svn/solrsharp
        Hide
        Jeff Rodenburg added a comment -

        The Example application includes some code for delete by query, but it is commented out. The particular code looks like:

        //List<QueryParameter> listQP = new List<QueryParameter>();
        //listQP.Add(new QueryParameter("id", "101"));
        //QueryParameterCollection queryParameterCollection = new QueryParameterCollection("delete", listQP);
        //Query query = new Query();
        //query.AddQueryParameters(queryParameterCollection, ParameterJoin.AND);
        //DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument(query);

        DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument("101");
        oUpdate.PostToIndex(deleteIndexDocument, true);

        Uncomment those and comment out the single DeleteIndexDocument("101") line to execute a delete-by-query.

        Show
        Jeff Rodenburg added a comment - The Example application includes some code for delete by query, but it is commented out. The particular code looks like: //List<QueryParameter> listQP = new List<QueryParameter>(); //listQP.Add(new QueryParameter("id", "101")); //QueryParameterCollection queryParameterCollection = new QueryParameterCollection("delete", listQP); //Query query = new Query(); //query.AddQueryParameters(queryParameterCollection, ParameterJoin.AND); //DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument(query); DeleteIndexDocument deleteIndexDocument = new DeleteIndexDocument("101"); oUpdate.PostToIndex(deleteIndexDocument, true); Uncomment those and comment out the single DeleteIndexDocument("101") line to execute a delete-by-query.
        Hide
        ms added a comment -

        Jeff- I will check the code at the link you provided. However, looking at the Examples, I do not see code sample for delete by query. My intention is to delte all documents/range of documents prior to (partially) reb-uilding the index.

        Show
        ms added a comment - Jeff- I will check the code at the link you provided. However, looking at the Examples, I do not see code sample for delete by query. My intention is to delte all documents/range of documents prior to (partially) reb-uilding the index.
        Hide
        Jeff Rodenburg added a comment -

        There was a bug in DeleteIndexDocument, which has been updated. Deleting by query vs. unique key was not being handled properly. The source at http://solrstuff.org/svn/solrsharp has been updated (documentation to updated later.)

        ms - for your scenario, this issues a delete-by-id request using the xml syntax "<delete><id>a</id></delete>". This is not the same as a delete-by-query, if that's the intention. Please look at the example for code that shows how to delete by query.

        Show
        Jeff Rodenburg added a comment - There was a bug in DeleteIndexDocument, which has been updated. Deleting by query vs. unique key was not being handled properly. The source at http://solrstuff.org/svn/solrsharp has been updated (documentation to updated later.) ms - for your scenario, this issues a delete-by-id request using the xml syntax "<delete><id> a </id></delete>". This is not the same as a delete-by-query, if that's the intention. Please look at the example for code that shows how to delete by query.
        Hide
        ms added a comment -

        Jeff,
        Thanks. I have it working against Solr 1.2. I am trying to figure out how to delete a document from the index. This is the code I am using and it is not working. TIA

        Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher)
        Dim doc As New org.apache.solr.SolrSharp.Indexing.DeleteIndexDocument
        doc.Id = "a"
        SolrU.PostToIndex(doc, True)
        SolrU.Optimize()

        Show
        ms added a comment - Jeff, Thanks. I have it working against Solr 1.2. I am trying to figure out how to delete a document from the index. This is the code I am using and it is not working. TIA Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher) Dim doc As New org.apache.solr.SolrSharp.Indexing.DeleteIndexDocument doc.Id = " a " SolrU.PostToIndex(doc, True) SolrU.Optimize()
        Hide
        Jeff Rodenburg added a comment -

        Solrsharp has been validated against the Solr 1.2 release. Validation was made using the example application that's available with the Solrsharp code.

        • The code has been moved to a new location and is now accessible via subversion. Many thanks to Ryan McKinley for hosting the codebase. You can find it at:

        http://solrstuff.org/svn/solrsharp

        • A new folder has been added: docs/api. We have MSDN-style documentation to help explain the full library. When you update from the repository, just point your browser to the local file at /docs/api/index.html.
        Show
        Jeff Rodenburg added a comment - Solrsharp has been validated against the Solr 1.2 release. Validation was made using the example application that's available with the Solrsharp code. The code has been moved to a new location and is now accessible via subversion. Many thanks to Ryan McKinley for hosting the codebase. You can find it at: http://solrstuff.org/svn/solrsharp A new folder has been added: docs/api. We have MSDN-style documentation to help explain the full library. When you update from the repository, just point your browser to the local file at /docs/api/index.html.
        Hide
        Jeff Rodenburg added a comment -

        Regarding ms IsValidUpdateIndexDocument issue:

        IsValidUpdateIndexDocument compares the fields of the referenced IndexDocument to the fields as read from the Solr configuration for the Solr instance you're updating. If the (non-copied) fields that are read from the solrconfig.xml file are not present in the list of fields from the IndexDocument, IsValidUpdateIndexDocument will fail.

        In your example, you should have only two fields defined in solrconfig.xml of the solr instance you're updating.

        Show
        Jeff Rodenburg added a comment - Regarding ms IsValidUpdateIndexDocument issue: IsValidUpdateIndexDocument compares the fields of the referenced IndexDocument to the fields as read from the Solr configuration for the Solr instance you're updating. If the (non-copied) fields that are read from the solrconfig.xml file are not present in the list of fields from the IndexDocument, IsValidUpdateIndexDocument will fail. In your example, you should have only two fields defined in solrconfig.xml of the solr instance you're updating.
        Hide
        Jeff Rodenburg added a comment -

        I'm not seeing comments that I've added via email response displayed on this issue, so I'll add them manually.

        Regarding question from Otis G. about repository access and 1.2 compliance:

        • I can't confirm right now (other project keeping me from this at the present.) On evaluation it doesn't appear that there are any breaking changes at this point, other than Lucene query syntax requiring a change for field values with semi-colons. However, there are new features in both this release as well as 1.1 that have not been incorporated into the library (defaults used as opposed to allowing choices where those exist.) I will update JIRA with a confirmation indicating the library is 1.2 compliant.
        • Repository access has been requested.
        Show
        Jeff Rodenburg added a comment - I'm not seeing comments that I've added via email response displayed on this issue, so I'll add them manually. Regarding question from Otis G. about repository access and 1.2 compliance: I can't confirm right now (other project keeping me from this at the present.) On evaluation it doesn't appear that there are any breaking changes at this point, other than Lucene query syntax requiring a change for field values with semi-colons. However, there are new features in both this release as well as 1.1 that have not been incorporated into the library (defaults used as opposed to allowing choices where those exist.) I will update JIRA with a confirmation indicating the library is 1.2 compliant. Repository access has been requested.
        Hide
        ms added a comment -

        I am having difficulty using SolrSharp in a VB.NET project. I am using .NET 2.0 uncer Windows Vista (VS 2005). The problem is that the document does not pass "IsValidUpdateIndexDocument'
        --------------------------------------------------------------
        I create a SolrSharp document thus:

        Imports System.Xml.Serialization
        Imports org.apache.solr.SolrSharp.Indexing

        <Serializable(), XmlRoot("add")> _
        Public Class SolrDocument

        Inherits UpdateIndexDocument

        Public Sub New()
        End Sub

        Public Sub New(ByVal id As String, ByVal Diagnosis As String)
        Me.Add(New IndexFieldValue("id", id))
        Me.Add(New IndexFieldValue("Diagnosis", Diagnosis))
        End Sub
        End Class

        Then I try to post it like this:

        Dim solrSearcher As SolrSharp.Configuration.SolrSearcher _
        = SolrSharp.Configuration.SolrSearchers.GetSearcher(SolrSharp.Configuration.Mode.Read)

        Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher)
        Dim doc As New SolrDocument(txtID.Text, txtDiagnosis.Text)

        If Not solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc) Then MessageBox.Show("Cannot validate document: ")

        Try

        SolrU.PostToIndex(doc, True)

        Catch ex As Exception
        MessageBox.Show(ex.ToString)
        End Try

        Show
        ms added a comment - I am having difficulty using SolrSharp in a VB.NET project. I am using .NET 2.0 uncer Windows Vista (VS 2005). The problem is that the document does not pass "IsValidUpdateIndexDocument' -------------------------------------------------------------- I create a SolrSharp document thus: Imports System.Xml.Serialization Imports org.apache.solr.SolrSharp.Indexing <Serializable(), XmlRoot("add")> _ Public Class SolrDocument Inherits UpdateIndexDocument Public Sub New() End Sub Public Sub New(ByVal id As String, ByVal Diagnosis As String) Me.Add(New IndexFieldValue("id", id)) Me.Add(New IndexFieldValue("Diagnosis", Diagnosis)) End Sub End Class Then I try to post it like this: Dim solrSearcher As SolrSharp.Configuration.SolrSearcher _ = SolrSharp.Configuration.SolrSearchers.GetSearcher(SolrSharp.Configuration.Mode.Read) Dim SolrU As New org.apache.solr.SolrSharp.Update.SolrUpdater(solrSearcher) Dim doc As New SolrDocument(txtID.Text, txtDiagnosis.Text) If Not solrSearcher.SolrSchema.IsValidUpdateIndexDocument(doc) Then MessageBox.Show("Cannot validate document: ") Try SolrU.PostToIndex(doc, True) Catch ex As Exception MessageBox.Show(ex.ToString) End Try
        Hide
        Otis Gospodnetic added a comment -

        Jeff & Co:
        What is the status of this as far as getting it into the repository?
        Something to do right after 1.2 is fully released?

        It's still in sync with 1.2 codebase and works, right Jeff?

        Show
        Otis Gospodnetic added a comment - Jeff & Co: What is the status of this as far as getting it into the repository? Something to do right after 1.2 is fully released? It's still in sync with 1.2 codebase and works, right Jeff?
        Hide
        Jeff Rodenburg added a comment -

        Full complement of files for distribution. Here's what's included:

        /src
        Source code for the SolrSharp project
        /docs
        MSDN-style web documentation of the SolrSharp library
        /example
        An example application that implements the library. Uses the sample configuration from Solr.

        More of this is forthcoming, but this is an initial release to get it in the hands of users.

        Show
        Jeff Rodenburg added a comment - Full complement of files for distribution. Here's what's included: /src Source code for the SolrSharp project /docs MSDN-style web documentation of the SolrSharp library /example An example application that implements the library. Uses the sample configuration from Solr. More of this is forthcoming, but this is an initial release to get it in the hands of users.
        Hide
        Jeff Rodenburg added a comment -

        This library and project consists of a few different chunks:

        /src - the source code for the library.
        /docs/api - MSDN-style documentation for the library (the MS world equivalent of javadoc).
        /example - working examples for the library.

        The namespace for the source code is "org.apache.solr.SolrSharp"

        Are there any documentation requirements for Apache compliance?

        One administrative note: I have a signed agreement on file with the Apache Foundation for code contributions, indemnification of my employer, etc. This is the standard doc that's required for all committers (I have committer status on the Lucene.Net project). I assume this applies for any Apache project.

        Show
        Jeff Rodenburg added a comment - This library and project consists of a few different chunks: /src - the source code for the library. /docs/api - MSDN-style documentation for the library (the MS world equivalent of javadoc). /example - working examples for the library. The namespace for the source code is "org.apache.solr.SolrSharp" Are there any documentation requirements for Apache compliance? One administrative note: I have a signed agreement on file with the Apache Foundation for code contributions, indemnification of my employer, etc. This is the standard doc that's required for all committers (I have committer status on the Lucene.Net project). I assume this applies for any Apache project.

          People

          • Assignee:
            Jeff Rodenburg
            Reporter:
            Jeff Rodenburg
          • Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:

              Development