Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      The solrconfig APIs ( SOLR-6607) now allow registering components through API. SOLR-6787 will support for blob storage.

      Jars should be able to be loaded from blobs

      example

       curl http://localhost:8983/solr/gettingstarted/config -H "Content-Type: application/json"  -d '{
      "create-requesthandler" : {"name" : "/mypath" ,
                                 "class":"org.apache.solr.handler.DumpRequestHandler",
                                 "lib" : "mycomponent",
                                 "version":2}
      }'
      
      
      1. SOLR-6801.patch
        38 kB
        Noble Paul
      2. SOLR-6801.patch
        20 kB
        Noble Paul

        Issue Links

          Activity

          Hide
          Noble Paul added a comment -

          Feature complete. No testcases yet.

          I will add the testcases and do some refactoring and commit this soon.

          comments/suggestions are welcome

          Show
          Noble Paul added a comment - Feature complete. No testcases yet. I will add the testcases and do some refactoring and commit this soon. comments/suggestions are welcome
          Hide
          Noble Paul added a comment -

          with testcases this time. I plan to commit this soon

          Show
          Noble Paul added a comment - with testcases this time. I plan to commit this soon
          Hide
          ASF subversion and git services added a comment -

          Commit 1646964 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1646964 ]

          SOLR-6801 Load RequestHandler from blob store

          Show
          ASF subversion and git services added a comment - Commit 1646964 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1646964 ] SOLR-6801 Load RequestHandler from blob store
          Hide
          ASF subversion and git services added a comment -

          Commit 1647018 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1647018 ]

          SOLR-6801: Fix formatting.

          Show
          ASF subversion and git services added a comment - Commit 1647018 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1647018 ] SOLR-6801 : Fix formatting.
          Hide
          ASF subversion and git services added a comment -

          Commit 1647034 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1647034 ]

          SOLR-6801 addressing test failures

          Show
          ASF subversion and git services added a comment - Commit 1647034 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1647034 ] SOLR-6801 addressing test failures
          Hide
          ASF subversion and git services added a comment -
          Show
          ASF subversion and git services added a comment - Commit 1647287 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1647287 ] SOLR-6801 fixing test failure http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2008/
          Hide
          ASF subversion and git services added a comment -

          Commit 1647539 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1647539 ]

          SOLR-6801 Load RequestHandler from blob store

          Show
          ASF subversion and git services added a comment - Commit 1647539 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1647539 ] SOLR-6801 Load RequestHandler from blob store
          Hide
          ASF subversion and git services added a comment -

          Commit 1647745 from Adrien Grand in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1647745 ]

          SOLR-6801: Fix javadoc error: parameter values expect a description.

          Show
          ASF subversion and git services added a comment - Commit 1647745 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1647745 ] SOLR-6801 : Fix javadoc error: parameter values expect a description.
          Hide
          ASF subversion and git services added a comment -

          Commit 1648689 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1648689 ]

          SOLR-6801 more tests , SOLR-6770 refactored code around

          Show
          ASF subversion and git services added a comment - Commit 1648689 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1648689 ] SOLR-6801 more tests , SOLR-6770 refactored code around
          Hide
          ASF subversion and git services added a comment -

          Commit 1648690 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1648690 ]

          SOLR-6801 setting svn props

          Show
          ASF subversion and git services added a comment - Commit 1648690 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1648690 ] SOLR-6801 setting svn props
          Hide
          ASF subversion and git services added a comment -

          Commit 1649345 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1649345 ]

          SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/

          Show
          ASF subversion and git services added a comment - Commit 1649345 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1649345 ] SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/
          Hide
          ASF subversion and git services added a comment -

          Commit 1649346 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1649346 ]

          SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/

          Show
          ASF subversion and git services added a comment - Commit 1649346 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1649346 ] SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/
          Hide
          ASF subversion and git services added a comment -

          Commit 1649348 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1649348 ]

          SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/

          Show
          ASF subversion and git services added a comment - Commit 1649348 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1649348 ] SOLR-6801 increase wait times to address failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1989/
          Hide
          Hoss Man added a comment -

          if you are still iterating to deal with repeated jenkins test failures, then it makes no sense to leave "Resolved" as the current status

          re-opening and setting as blocker (if we can't get the tests stable on the 5x branch, then this should be ripped out before the 5.0 release)

          Show
          Hoss Man added a comment - if you are still iterating to deal with repeated jenkins test failures, then it makes no sense to leave "Resolved" as the current status re-opening and setting as blocker (if we can't get the tests stable on the 5x branch, then this should be ripped out before the 5.0 release)
          Hide
          Noble Paul added a comment -

          I could never get this test to fail in my local box. Sometimes , the only choice is to debug the logs from the test machines.

          Show
          Noble Paul added a comment - I could never get this test to fail in my local box. Sometimes , the only choice is to debug the logs from the test machines.
          Hide
          ASF subversion and git services added a comment -

          Commit 1649821 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1649821 ]

          SOLR-6801 addressing test failures, SOLR-6607 supporting all initArgs for requesthandlers

          Show
          ASF subversion and git services added a comment - Commit 1649821 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1649821 ] SOLR-6801 addressing test failures, SOLR-6607 supporting all initArgs for requesthandlers
          Hide
          ASF subversion and git services added a comment -

          Commit 1649826 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1649826 ]

          SOLR-6801 addressing test failures, SOLR-6607 supporting all initArgs for requesthandlers

          Show
          ASF subversion and git services added a comment - Commit 1649826 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1649826 ] SOLR-6801 addressing test failures, SOLR-6607 supporting all initArgs for requesthandlers
          Hide
          ASF subversion and git services added a comment -

          Commit 1649877 from shalin@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1649877 ]

          SOLR-6801: Use static random initialized with random.seed and use it in Collections.shuffle to make forbidden-api checks pass

          Show
          ASF subversion and git services added a comment - Commit 1649877 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1649877 ] SOLR-6801 : Use static random initialized with random.seed and use it in Collections.shuffle to make forbidden-api checks pass
          Hide
          ASF subversion and git services added a comment -

          Commit 1649888 from shalin@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1649888 ]

          SOLR-6801: Use static random initialized with random.seed and use it in Collections.shuffle to make forbidden-api checks pass

          Show
          ASF subversion and git services added a comment - Commit 1649888 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1649888 ] SOLR-6801 : Use static random initialized with random.seed and use it in Collections.shuffle to make forbidden-api checks pass
          Hide
          ASF subversion and git services added a comment -

          Commit 1652430 from Noble Paul in branch 'dev/branches/lucene_solr_5_0'
          [ https://svn.apache.org/r1652430 ]

          SOLR-6801 NPE on core reload

          Show
          ASF subversion and git services added a comment - Commit 1652430 from Noble Paul in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1652430 ] SOLR-6801 NPE on core reload
          Hide
          ASF subversion and git services added a comment -

          Commit 1652431 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1652431 ]

          SOLR-6801 NPE on core reload

          Show
          ASF subversion and git services added a comment - Commit 1652431 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1652431 ] SOLR-6801 NPE on core reload
          Hide
          ASF subversion and git services added a comment -

          Commit 1652432 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1652432 ]

          SOLR-6801 NPE on core reload

          Show
          ASF subversion and git services added a comment - Commit 1652432 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1652432 ] SOLR-6801 NPE on core reload
          Hide
          ASF subversion and git services added a comment -

          Commit 1652442 from Noble Paul in branch 'dev/branches/lucene_solr_5_0'
          [ https://svn.apache.org/r1652442 ]

          SOLR-6801 version can be string or number

          Show
          ASF subversion and git services added a comment - Commit 1652442 from Noble Paul in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1652442 ] SOLR-6801 version can be string or number
          Hide
          ASF subversion and git services added a comment -

          Commit 1652444 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1652444 ]

          SOLR-6801 version can be string or number

          Show
          ASF subversion and git services added a comment - Commit 1652444 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1652444 ] SOLR-6801 version can be string or number
          Hide
          ASF subversion and git services added a comment -

          Commit 1652445 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1652445 ]

          SOLR-6801 version can be string or number

          Show
          ASF subversion and git services added a comment - Commit 1652445 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1652445 ] SOLR-6801 version can be string or number
          Hide
          Anshum Gupta added a comment -

          Noble Paul is this all wrapped up w.r.t. 5.0?

          Show
          Anshum Gupta added a comment - Noble Paul is this all wrapped up w.r.t. 5.0?
          Hide
          ASF subversion and git services added a comment -

          Commit 1653322 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1653322 ]

          SOLR-6801 test always writes to leader so that replication lag does not impact next insertion

          Show
          ASF subversion and git services added a comment - Commit 1653322 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1653322 ] SOLR-6801 test always writes to leader so that replication lag does not impact next insertion
          Hide
          ASF subversion and git services added a comment -

          Commit 1653324 from Noble Paul in branch 'dev/branches/lucene_solr_5_0'
          [ https://svn.apache.org/r1653324 ]

          SOLR-6801 test always writes to leader so that replication lag does not impact next insertion

          Show
          ASF subversion and git services added a comment - Commit 1653324 from Noble Paul in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1653324 ] SOLR-6801 test always writes to leader so that replication lag does not impact next insertion
          Hide
          Noble Paul added a comment -

          OK Anshum Gupta it should be fine now. However I shall wait for a day before marking this as resolved

          Show
          Noble Paul added a comment - OK Anshum Gupta it should be fine now. However I shall wait for a day before marking this as resolved
          Hide
          ASF subversion and git services added a comment -

          Commit 1653328 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1653328 ]

          SOLR-6801 test always writes to leader so that replication lag does not impact next insertion

          Show
          ASF subversion and git services added a comment - Commit 1653328 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1653328 ] SOLR-6801 test always writes to leader so that replication lag does not impact next insertion
          Hide
          Hoss Man added a comment -

          I don't fully understand this functionality, but this commit looks scary as shit...

          SOLR-6801 test always writes to leader so that replication lag does not impact next insertion

          ...why this change is considered a "safe" solution to the existing test failures? .. it seems to just be making the test absurdly week – isn't the root problem here (replication lag) something that can and will come up when end users try to use this same functionality?

          if the only way the test can reliably pass is if we put hacks into the test to ensure that the updates only go to the "blob" leader, that suggests to me that the functionality itself isn't going to work reliably for end users unless they also only ever hit the leader ... what stops a user from encountering the same replication lag?

          it seems like either:

          1) we need to protect user by locking the feature down:

          • document that the blob store it only works when talking to the "blob" leader
          • lock down the blob handler to reject requests to nodes that aren't the leader

          ...OR...

          2) the solr code itself needs hardened to do some sort of forward to leader (ala: atomic updates and/or real time get) or push the responsibilty down to the client via something like opportunistic locking (i'm hand wavy here because i don't fully understand the usecases/goals)

          Show
          Hoss Man added a comment - I don't fully understand this functionality, but this commit looks scary as shit... SOLR-6801 test always writes to leader so that replication lag does not impact next insertion ...why this change is considered a "safe" solution to the existing test failures? .. it seems to just be making the test absurdly week – isn't the root problem here (replication lag) something that can and will come up when end users try to use this same functionality? if the only way the test can reliably pass is if we put hacks into the test to ensure that the updates only go to the "blob" leader, that suggests to me that the functionality itself isn't going to work reliably for end users unless they also only ever hit the leader ... what stops a user from encountering the same replication lag? it seems like either: 1) we need to protect user by locking the feature down: document that the blob store it only works when talking to the "blob" leader lock down the blob handler to reject requests to nodes that aren't the leader ...OR... 2) the solr code itself needs hardened to do some sort of forward to leader (ala: atomic updates and/or real time get) or push the responsibilty down to the client via something like opportunistic locking (i'm hand wavy here because i don't fully understand the usecases/goals)
          Hide
          Noble Paul added a comment -

          Actually this commit belonged to SOLR-6787. This ticket has nothing to do with this failure.
          So this reply must go there

          BlobHandler works as follows

          • find out the latest version in the index now and write a newer version to the index
          • the test verifies if the next version is created .
          • When I write to /.system/blob it may go to a replica and if the replica hasn't got the update yet it can write to an older version and the test comparison fails

          atomic updates are not a solution because we are not updating the same doc. realtime get is a possible solution .

          Show
          Noble Paul added a comment - Actually this commit belonged to SOLR-6787 . This ticket has nothing to do with this failure. So this reply must go there BlobHandler works as follows find out the latest version in the index now and write a newer version to the index the test verifies if the next version is created . When I write to /.system/blob it may go to a replica and if the replica hasn't got the update yet it can write to an older version and the test comparison fails atomic updates are not a solution because we are not updating the same doc. realtime get is a possible solution .
          Hide
          ASF subversion and git services added a comment -

          Commit 1653451 from Noble Paul in branch 'dev/branches/lucene_solr_5_0'
          [ https://svn.apache.org/r1653451 ]

          SOLR-6787, SOLR-6801 use realtime get to verify that the versions do not collide

          Show
          ASF subversion and git services added a comment - Commit 1653451 from Noble Paul in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1653451 ] SOLR-6787 , SOLR-6801 use realtime get to verify that the versions do not collide
          Hide
          ASF subversion and git services added a comment -

          Commit 1653452 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1653452 ]

          SOLR-6787, SOLR-6801 use realtime get to verify that the versions do not collide

          Show
          ASF subversion and git services added a comment - Commit 1653452 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1653452 ] SOLR-6787 , SOLR-6801 use realtime get to verify that the versions do not collide
          Hide
          ASF subversion and git services added a comment -

          Commit 1653453 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1653453 ]

          SOLR-6787, SOLR-6801 use realtime get to verify that the versions do not collide

          Show
          ASF subversion and git services added a comment - Commit 1653453 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1653453 ] SOLR-6787 , SOLR-6801 use realtime get to verify that the versions do not collide
          Hide
          Anshum Gupta added a comment -

          Noble Paul Is this resolved?

          Show
          Anshum Gupta added a comment - Noble Paul Is this resolved?
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Noble Paul
              Reporter:
              Noble Paul
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development