Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 4.9, 5.0
    • Component/s: None
    • Labels:
      None

      Description

      Marko Bonaci has updated the WIKI page to add the DataImportScheduler, but I cannot find a JIRA ticket for it?

      http://wiki.apache.org/solr/DataImportHandler

      Do we have a ticket so the code can be tracked?

      1. patch.txt
        21 kB
        Marko Bonaci
      2. SOLR-2305-1.diff
        20 kB
        Esteve Fernandez

        Issue Links

          Activity

          Hide
          Uwe Schindler added a comment -

          Move issue to Solr 4.9.

          Show
          Uwe Schindler added a comment - Move issue to Solr 4.9.
          Hide
          James Pace added a comment -

          What is the status of this? Are we just waiting to get enough votes for it to be committed?

          Show
          James Pace added a comment - What is the status of this? Are we just waiting to get enough votes for it to be committed?
          Hide
          Steve Rowe added a comment -

          Bulk move 4.4 issues to 4.5 and 5.0

          Show
          Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
          Hide
          Smita Raval added a comment - - edited

          I would also vote for DIH scheduler.
          It would be really nice to add enable parameter to DIH defaults, similar to replicationHandler.
          So, can keep the same config for master and slaves and triggering dataimport only if master is enabled.

          Show
          Smita Raval added a comment - - edited I would also vote for DIH scheduler. It would be really nice to add enable parameter to DIH defaults, similar to replicationHandler. So, can keep the same config for master and slaves and triggering dataimport only if master is enabled.
          Hide
          Marko Bonaci added a comment -

          Otis Gospodnetic
          Got it! Will do...

          Show
          Marko Bonaci added a comment - Otis Gospodnetic Got it! Will do...
          Hide
          Otis Gospodnetic added a comment -

          Marko Bonaci comments:

          • please remove @author - long ago we decided to get rid of that
          • please use matching code style (I think there are Eclipse settings on HowToContribute on Wiki or some place like that)
          • please add a unit test if at all possible, otherwise somebody as to take a lot of time to set things up and try this – the easier you make it for people to understand the code and make sure it works as advertised the higher the chances somebody will look at it – comes with OSS
          Show
          Otis Gospodnetic added a comment - Marko Bonaci comments: please remove @author - long ago we decided to get rid of that please use matching code style (I think there are Eclipse settings on HowToContribute on Wiki or some place like that) please add a unit test if at all possible, otherwise somebody as to take a lot of time to set things up and try this – the easier you make it for people to understand the code and make sure it works as advertised the higher the chances somebody will look at it – comes with OSS
          Hide
          Marko Bonaci added a comment -

          Erick Erickson
          Isn't the number of votes enough to push the issue? It's 14th from the top.

          I assume that, before the patch gets commited someone more experienced in Solr source standards should check the code.
          Is it time to change the Assignee to myself and Status to 'Ready To Review'?

          Thanks

          Show
          Marko Bonaci added a comment - Erick Erickson Isn't the number of votes enough to push the issue? It's 14th from the top. I assume that, before the patch gets commited someone more experienced in Solr source standards should check the code. Is it time to change the Assignee to myself and Status to 'Ready To Review'? Thanks
          Hide
          Erick Erickson added a comment -

          Not quite. That's usually just a marker that 4.1 is the earliest that it could be included. Somebody needs to get behind this and nudge the committers to have it actually make it into 4.1.

          Show
          Erick Erickson added a comment - Not quite. That's usually just a marker that 4.1 is the earliest that it could be included. Somebody needs to get behind this and nudge the committers to have it actually make it into 4.1.
          Hide
          Marko Bonaci added a comment - - edited

          Billy Correct me if I'm wrong, but if I read the header of the issue correctly, it's planned to be included in 4.1 (affects version 4 which then becomes version 4.1).

          Show
          Marko Bonaci added a comment - - edited Billy Correct me if I'm wrong, but if I read the header of the issue correctly, it's planned to be included in 4.1 (affects version 4 which then becomes version 4.1).
          Hide
          Marko Bonaci added a comment - - edited

          Can I please ask someone (whoever has the appropriate access level) to remove my name from the title of this issue.
          I normally am an egocentric guy, but this, even for me, is slightly over the top

          Thanks

          Show
          Marko Bonaci added a comment - - edited Can I please ask someone (whoever has the appropriate access level) to remove my name from the title of this issue. I normally am an egocentric guy, but this, even for me, is slightly over the top Thanks
          Hide
          Billy added a comment - - edited

          Are there still plans to add this to the the version 4 distro? I don't see it in the 4.0.0-BETA distro yet. IMHO, I see great benefit for adding this, please consider. Thanks!

          Show
          Billy added a comment - - edited Are there still plans to add this to the the version 4 distro? I don't see it in the 4.0.0-BETA distro yet. IMHO, I see great benefit for adding this, please consider. Thanks!
          Hide
          Marko Bonaci added a comment -

          Until DIH Scheduler is included in official Solr distribution you can use JAR file I published here, on Google code.

          Show
          Marko Bonaci added a comment - Until DIH Scheduler is included in official Solr distribution you can use JAR file I published here , on Google code.
          Hide
          Arcadius Ahouansou added a comment -

          A DIH scheduler is highly needed.

          Many end up implementing their own or use Cron.

          In our case, we ended up writing a small scheduler web app that does HTTP Get to trigger the import.

          It would be really useful if this was included in the official solr distribution.

          Thanks.

          Show
          Arcadius Ahouansou added a comment - A DIH scheduler is highly needed. Many end up implementing their own or use Cron. In our case, we ended up writing a small scheduler web app that does HTTP Get to trigger the import. It would be really useful if this was included in the official solr distribution. Thanks.
          Hide
          Marko Bonaci added a comment -

          I'd certainly be very proud if my small contribution gets included in the trunk.
          It's my ultimate dream to work on an open source project full time, but "corporate OS" is still in its infancy here, in Croatia
          We'll get there... In the mean time, I'm here only when I can convince my bosses that OS is the way to go. That's how I got into Solr.

          Thank you for the info.

          Show
          Marko Bonaci added a comment - I'd certainly be very proud if my small contribution gets included in the trunk. It's my ultimate dream to work on an open source project full time, but "corporate OS" is still in its infancy here, in Croatia We'll get there... In the mean time, I'm here only when I can convince my bosses that OS is the way to go. That's how I got into Solr. Thank you for the info.
          Hide
          Anders Thulin added a comment -

          I would like to see the scheduler included very much! (vote)

          Show
          Anders Thulin added a comment - I would like to see the scheduler included very much! (vote)
          Hide
          Erick Erickson added a comment -

          The discussions are right here attached to the JIRA, one of the principles of open source is that decisions are reached in public.

          The general process is that someone has a suggestion and/or code and makes a JIRA. That doesn't guarantee that it'll be included, it's a matter of the idea generating enough general interest to be committed. If you think this is valuable to the larger community, you need to get behind it and push.

          Show
          Erick Erickson added a comment - The discussions are right here attached to the JIRA, one of the principles of open source is that decisions are reached in public. The general process is that someone has a suggestion and/or code and makes a JIRA. That doesn't guarantee that it'll be included, it's a matter of the idea generating enough general interest to be committed. If you think this is valuable to the larger community, you need to get behind it and push.
          Hide
          Marko Bonaci added a comment -

          I don't need the code.
          I ask because I was wondering why it wasn't included in Solr?

          Is the code discarded (it'll never get in) or is it waiting for something?
          Are discussions on whether to include something or not open?

          Show
          Marko Bonaci added a comment - I don't need the code. I ask because I was wondering why it wasn't included in Solr? Is the code discarded (it'll never get in) or is it waiting for something? Are discussions on whether to include something or not open?
          Hide
          Okke Klein added a comment -

          It's not committed to trunk. To get the code apply the SOLR-2305-1.diff patch

          Show
          Okke Klein added a comment - It's not committed to trunk. To get the code apply the SOLR-2305 -1.diff patch
          Hide
          Marko Bonaci added a comment -

          Hi Esteve,
          I cannot find the scheduler source in Solr trunk.
          I looked here
          http://svn.apache.org/repos/asf/lucene/dev/trunk/solr/contrib/

          Where the source you updated went
          Can you please post the link.

          Thanks

          Show
          Marko Bonaci added a comment - Hi Esteve, I cannot find the scheduler source in Solr trunk. I looked here http://svn.apache.org/repos/asf/lucene/dev/trunk/solr/contrib/ Where the source you updated went Can you please post the link. Thanks
          Hide
          Esteve Fernandez added a comment - - edited

          The patch I submitted (SOLR-2305-1.diff) is updated to Solr trunk, adds a listener to web.xml and fixes a bug in the hardcoded path to dataimport.properties that prevented it from running on Unix systems.

          Show
          Esteve Fernandez added a comment - - edited The patch I submitted ( SOLR-2305 -1.diff) is updated to Solr trunk, adds a listener to web.xml and fixes a bug in the hardcoded path to dataimport.properties that prevented it from running on Unix systems.
          Hide
          Marko Bonaci added a comment - - edited

          This is patch for adding DIHScheduler v1.2 to Solr.
          I didn't know I could make a patch concerning only org.apache.solr.handler.dataimport package
          So finally, here it is.

          Since I still have problems with build path/packages in Eclipse:
          Wasn't tested at all.
          No unit tests.
          Whoever will be adding this please feel free to contact me if such a need arises.
          Also, all criticism is more than welcome, I want to learn to do this the right way.

          Thanks

          Show
          Marko Bonaci added a comment - - edited This is patch for adding DIHScheduler v1.2 to Solr. I didn't know I could make a patch concerning only org.apache.solr.handler.dataimport package So finally, here it is. Since I still have problems with build path/packages in Eclipse: Wasn't tested at all. No unit tests. Whoever will be adding this please feel free to contact me if such a need arises. Also, all criticism is more than welcome, I want to learn to do this the right way. Thanks
          Hide
          Marko Bonaci added a comment -

          Hi Bill,
          I had difficulties with setting up the project in Eclipse, and although I have successfully done it in the end, I think that the patch file wont be usable (due to many build path changes I made)?

          All you have to do to incorporate DIHScheduler is to follow the instructions I posted here:
          http://wiki.apache.org/solr/DataImportHandler#Scheduling

          If you run into any kind of problem feel free to post the question here and I'll try to respond promptly.

          Thank you.

          Show
          Marko Bonaci added a comment - Hi Bill, I had difficulties with setting up the project in Eclipse, and although I have successfully done it in the end, I think that the patch file wont be usable (due to many build path changes I made)? All you have to do to incorporate DIHScheduler is to follow the instructions I posted here: http://wiki.apache.org/solr/DataImportHandler#Scheduling If you run into any kind of problem feel free to post the question here and I'll try to respond promptly. Thank you.
          Hide
          Marko Bonaci added a comment -

          I'll attach the patch during the following weekend.

          Show
          Marko Bonaci added a comment - I'll attach the patch during the following weekend.
          Hide
          Bill Bell added a comment -

          How do we get the scheduler added to SOLR ?

          Show
          Bill Bell added a comment - How do we get the scheduler added to SOLR ?
          Hide
          Bill Bell added a comment -

          The best link: http://wiki.apache.org/solr/HowToContribute

          Get the latest TRUNK code, change the files, and generate PATCH file. Upload to this ticket in JIRA (attach file). Do it at the top of the directory structure.

          svn diff > SOLR-2305.patch

          Show
          Bill Bell added a comment - The best link: http://wiki.apache.org/solr/HowToContribute Get the latest TRUNK code, change the files, and generate PATCH file. Upload to this ticket in JIRA (attach file). Do it at the top of the directory structure. svn diff > SOLR-2305 .patch
          Hide
          Marko Bonaci added a comment -

          I'd like to help, but you'll have to explain me how to do it.
          What needs to be prepared? java files? the whole project?
          Do I have to check out the project in Eclipse? And, if yes, how to get commit access rights?

          I've never contributed before, obviously, but I'm interested in learning how to do it properly... Links?

          Show
          Marko Bonaci added a comment - I'd like to help, but you'll have to explain me how to do it. What needs to be prepared? java files? the whole project? Do I have to check out the project in Eclipse? And, if yes, how to get commit access rights? I've never contributed before, obviously, but I'm interested in learning how to do it properly... Links?

            People

            • Assignee:
              Unassigned
              Reporter:
              Bill Bell
            • Votes:
              19 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:

                Development