Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.1, 2.9.0
    • Fix Version/s: 2.9.0
    • Component/s: None
    • Labels:
    • Patch Info:
      Patch Available
    • Estimated Complexity:
      Unknown

      Description

      A Solr component for Camel. This component should support both indexing and querying, with flexibility in how these two features operate.

      1. CAMEL-4539.patch
        324 kB
        Alex Cowell
      2. camel-solr.patch
        180 kB
        Ben O'Day
      3. tutorial.pdf
        52 kB
        Bilgin Ibryam

        Issue Links

          Activity

          Hide
          Alex Cowell added a comment -

          This patch adds a Solr component to Camel, which currently just supports indexing.

          Users can configure how data is mapped to fields in their Solr schema by setting headers on the message. Any request handler can be specified as an Endpoint option (the default handler for XML; CSV handler; extracting handler for PDF, DOC etc.), so most common filetypes are supported.

          This component uses SolrJ to connect to a Solr instance and the underlying API is exposed to allow users to configure the Solr connection on the Endpoint.

          All the work is done by a Producer as it didn't seem necessary to create a Consumer specifically for Solr when an HTTP endpoint could do the job just as well.

          This patch was made by Alex Cowell and Bilgin Ibryam and we are grateful to our employer, Sourcesense, for letting us contribute to this project during company time.

          Show
          Alex Cowell added a comment - This patch adds a Solr component to Camel, which currently just supports indexing. Users can configure how data is mapped to fields in their Solr schema by setting headers on the message. Any request handler can be specified as an Endpoint option (the default handler for XML; CSV handler; extracting handler for PDF, DOC etc.), so most common filetypes are supported. This component uses SolrJ to connect to a Solr instance and the underlying API is exposed to allow users to configure the Solr connection on the Endpoint. All the work is done by a Producer as it didn't seem necessary to create a Consumer specifically for Solr when an HTTP endpoint could do the job just as well. This patch was made by Alex Cowell and Bilgin Ibryam and we are grateful to our employer, Sourcesense, for letting us contribute to this project during company time.
          Hide
          Ben O'Day added a comment -

          was happy to come across this, I'm going to review it soon...

          Show
          Ben O'Day added a comment - was happy to come across this, I'm going to review it soon...
          Hide
          Bilgin Ibryam added a comment -

          Thanks in advance for trying it out Ben.

          Show
          Bilgin Ibryam added a comment - Thanks in advance for trying it out Ben.
          Hide
          Ben O'Day added a comment -

          Bilgin, can you attach the src/test/resources/data/tutorial.pdf file, thanks...

          Show
          Ben O'Day added a comment - Bilgin, can you attach the src/test/resources/data/tutorial.pdf file, thanks...
          Hide
          Bilgin Ibryam added a comment -

          Here it is

          Show
          Bilgin Ibryam added a comment - Here it is
          Hide
          Ben O'Day added a comment -

          Bilgin, thanks. At first glance, all seems good...will take some time to review it all and get it committed (need to discuss on DEV list, etc).

          Off the top, I'm planning on making the following changes to round it out a bit, let me know your thoughts...

          -add support for the following operations: OPTIMIZE, DELETE_BY_ID, DELETE_BY_QUERY (possibly ADD, ADD_BEAN)
          -rename SolrHeaders.java to SolrConstants.java
          -prefix header values with "Solr" and make mix case ("SolrOperationInsert", etc)

          Anything else you guys need in the near term?

          Show
          Ben O'Day added a comment - Bilgin, thanks. At first glance, all seems good...will take some time to review it all and get it committed (need to discuss on DEV list, etc). Off the top, I'm planning on making the following changes to round it out a bit, let me know your thoughts... -add support for the following operations: OPTIMIZE, DELETE_BY_ID, DELETE_BY_QUERY (possibly ADD, ADD_BEAN) -rename SolrHeaders.java to SolrConstants.java -prefix header values with "Solr" and make mix case ("SolrOperationInsert", etc) Anything else you guys need in the near term?
          Hide
          Bilgin Ibryam added a comment -

          Would be good to have also these features in the component. Also the other changes seems right, so go on.

          Show
          Bilgin Ibryam added a comment - Would be good to have also these features in the component. Also the other changes seems right, so go on.
          Hide
          Ben O'Day added a comment -

          attached is an updated patch that adds the following:

          -new operations: ADD_BEAN, DELETE_BY_ID, DELETE_BY_QUERY, OPTIMIZE
          -tweaks to pom.xml to support OSGI
          -added feature to karaf features.xml

          I'm still trying to get this working with OSGI at the moment, but wanted to at least put this up for any general feedback...

          Show
          Ben O'Day added a comment - attached is an updated patch that adds the following: -new operations: ADD_BEAN, DELETE_BY_ID, DELETE_BY_QUERY, OPTIMIZE -tweaks to pom.xml to support OSGI -added feature to karaf features.xml I'm still trying to get this working with OSGI at the moment, but wanted to at least put this up for any general feedback...
          Hide
          Ben O'Day added a comment -

          updated patch for review that works in Karaf (uses wrap:mvn dependency until SMX bundle is created for SolrJ)

          Show
          Ben O'Day added a comment - updated patch for review that works in Karaf (uses wrap:mvn dependency until SMX bundle is created for SolrJ)
          Hide
          Jean-Baptiste Onofré added a comment -

          The feature descriptor doesn't look good to me:
          1/ regarding the solr pom, the httpclient version to use is 3.x, not 4.x
          2/ the lucene bundle is require to get the org.apache.lucene.util package

          I will submit a patch with the correct feature descriptor.

          Show
          Jean-Baptiste Onofré added a comment - The feature descriptor doesn't look good to me: 1/ regarding the solr pom, the httpclient version to use is 3.x, not 4.x 2/ the lucene bundle is require to get the org.apache.lucene.util package I will submit a patch with the correct feature descriptor.
          Hide
          Ben O'Day added a comment -

          I just committed this component, see revision: 1209333.

          JBO, I removed the unnecessary references to httpclient. I deployed/tested this successfully in Karaf 2.2.4. Please review the karaf/features.xml. In particular, I used "<bundle dependency="true">wrap:mvn:org.apache.solr/solr-solrj/$

          {solr-version}

          </bundle>" until the SMX bundle is available.

          Otherwise, I'll start working on the confluence docs tonight.

          Let me know if there are any issues...thanks

          Show
          Ben O'Day added a comment - I just committed this component, see revision: 1209333. JBO, I removed the unnecessary references to httpclient. I deployed/tested this successfully in Karaf 2.2.4. Please review the karaf/features.xml. In particular, I used "<bundle dependency="true">wrap:mvn:org.apache.solr/solr-solrj/$ {solr-version} </bundle>" until the SMX bundle is available. Otherwise, I'll start working on the confluence docs tonight. Let me know if there are any issues...thanks
          Hide
          Alex Cowell added a comment -

          Good work Ben!

          I've got some additional documentation from when we were using this component internally which I could add to Confluence if you'd like?

          Show
          Alex Cowell added a comment - Good work Ben! I've got some additional documentation from when we were using this component internally which I could add to Confluence if you'd like?
          Hide
          Ben O'Day added a comment -

          Alex, sounds good...I just added the basics to get the ball rolling. Also,
          if there are other features that you guys would like to see added, log a
          new JIRA for it...

          On Sun, Dec 4, 2011 at 10:05 AM, Alex Cowell (Commented) (JIRA) <

          Show
          Ben O'Day added a comment - Alex, sounds good...I just added the basics to get the ball rolling. Also, if there are other features that you guys would like to see added, log a new JIRA for it... On Sun, Dec 4, 2011 at 10:05 AM, Alex Cowell (Commented) (JIRA) <

            People

            • Assignee:
              Ben O'Day
              Reporter:
              Alex Cowell
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development