Solr
  1. Solr
  2. SOLR-5922

Add Properties and other parameters to SolrJ Collection Admin Request calls

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: SolrJ
    • Labels:
      None

      Description

      SOLR-5208 added functionality for the setting of core.properties key/values at create-time on Collections API.

      We should allow the same behaviour for SolrJ API calls as well.

      Also I want to add get and set methods to be able to add 'instanceDir', 'dataDir', 'ulogDir' for a create colleciton call.

      1. SOLR-5922.patch
        11 kB
        Varun Thacker
      2. SOLR-5922.patch
        9 kB
        Varun Thacker
      3. SOLR-5922.patch
        11 kB
        Varun Thacker
      4. SOLR-5922.patch
        11 kB
        Varun Thacker
      5. SOLR-5922.patch
        12 kB
        Varun Thacker
      6. SOLR-5922.patch
        9 kB
        Varun Thacker
      7. SOLR-5922.patch
        4 kB
        Varun Thacker

        Issue Links

          Activity

          Hide
          Varun Thacker added a comment -
          • For createCollection it adds 3 params - ulogDir, instanceDir and dataDir
          • Adds options to add Properties to both createCollection and splitShard methods
          • User does not have to prefix "property." when defining properties. It is done internally.
          Show
          Varun Thacker added a comment - For createCollection it adds 3 params - ulogDir, instanceDir and dataDir Adds options to add Properties to both createCollection and splitShard methods User does not have to prefix "property." when defining properties. It is done internally.
          Hide
          Shalin Shekhar Mangar added a comment -

          Thanks Varun! Looks good. Can you add a test as well?

          Show
          Shalin Shekhar Mangar added a comment - Thanks Varun! Looks good. Can you add a test as well?
          Hide
          Varun Thacker added a comment -
          • New patch with a test case. Test is broken when it tries to verify if the properties is getting used correctly. Any ideas on what I am doing incorrectly?
          • Fixed typo in CoreAdminCreateDiscoveryTest
          Show
          Varun Thacker added a comment - New patch with a test case. Test is broken when it tries to verify if the properties is getting used correctly. Any ideas on what I am doing incorrectly? Fixed typo in CoreAdminCreateDiscoveryTest
          Hide
          Shalin Shekhar Mangar added a comment -

          Test is broken when it tries to verify if the properties is getting used correctly. Any ideas on what I am doing incorrectly?

          Thanks Varun. I don't think initCore is the right way. Why don't you make a core admin status call against a replica of the newly created collection? (hint: use the new clusterstatus API to know the baseUrl of the replica)

          Show
          Shalin Shekhar Mangar added a comment - Test is broken when it tries to verify if the properties is getting used correctly. Any ideas on what I am doing incorrectly? Thanks Varun. I don't think initCore is the right way. Why don't you make a core admin status call against a replica of the newly created collection? (hint: use the new clusterstatus API to know the baseUrl of the replica)
          Hide
          Varun Thacker added a comment -

          New patch with 2 test cases.

          • I only see these 2 directories getting created
            1. testInstanceDirAsPropertyParam-XYZ/
            2. testInstanceDirAsPropertyParam-XYZ/testuLogDirAsPropertyParam-XYZ/

          Don't see the dataDir getting created

          Show
          Varun Thacker added a comment - New patch with 2 test cases. I only see these 2 directories getting created 1. testInstanceDirAsPropertyParam-XYZ/ 2. testInstanceDirAsPropertyParam-XYZ/testuLogDirAsPropertyParam-XYZ/ Don't see the dataDir getting created
          Hide
          Varun Thacker added a comment -

          New patch. The test case gets the base_url and uses that to create a server to call the CoreAdmin STATUS API call.

          I still don't see the data_dir being created in the filesystem. The STATUS call returns the correct location to the data_dir thought.

          instance_dir, uLog and core.properties gets created correctly.

          Show
          Varun Thacker added a comment - New patch. The test case gets the base_url and uses that to create a server to call the CoreAdmin STATUS API call. I still don't see the data_dir being created in the filesystem. The STATUS call returns the correct location to the data_dir thought. instance_dir, uLog and core.properties gets created correctly.
          Hide
          Anshum Gupta added a comment -

          Varun Thacker Can you provide an updated patch that doesn't really try to do anything at the coreadmin level (even for testing) and only uses the new way of using the CollectionAdminRequest

          New way of using SolrJ for Collections API
          Create createRequest = new Create()
          createRequest.setCollectionName(name);
          createRequest.setConfigName(..);
          .
          .
          

          I'll review and commit it once you provide an update.

          Show
          Anshum Gupta added a comment - Varun Thacker Can you provide an updated patch that doesn't really try to do anything at the coreadmin level (even for testing) and only uses the new way of using the CollectionAdminRequest New way of using SolrJ for Collections API Create createRequest = new Create() createRequest.setCollectionName(name); createRequest.setConfigName(..); . . I'll review and commit it once you provide an update.
          Hide
          Varun Thacker added a comment -

          New patch.

          Since our motivation here is to make sure that whatever is supported in the Collection API calls should be mirrored for SolrJ I just added the setProperties() option for Create and SplitShard calls.

          Patch removes the additional convenience methods in previous patches around uLogDir, dataDir and instanceDir as those are not directly exposed through the collections api call but only through property.key=value format.

          Show
          Varun Thacker added a comment - New patch. Since our motivation here is to make sure that whatever is supported in the Collection API calls should be mirrored for SolrJ I just added the setProperties() option for Create and SplitShard calls. Patch removes the additional convenience methods in previous patches around uLogDir, dataDir and instanceDir as those are not directly exposed through the collections api call but only through property.key=value format.
          Hide
          Varun Thacker added a comment - - edited

          Patch which improves how the test fetches the replica.

          Show
          Varun Thacker added a comment - - edited Patch which improves how the test fetches the replica.
          Hide
          Anshum Gupta added a comment -

          CREATE, SPLITSHARD, MIGRATE, CREATESHARD, ADDREPLICA, ADDREPLICAPROP are Collection API calls that support setting of property.* value for a core. Can you have all of them support it via SolrJ i.e. expand it to include all of the above?
          The rest looks fine to me. Once you update it, I should be able to commit this.

          Show
          Anshum Gupta added a comment - CREATE, SPLITSHARD, MIGRATE, CREATESHARD, ADDREPLICA, ADDREPLICAPROP are Collection API calls that support setting of property.* value for a core. Can you have all of them support it via SolrJ i.e. expand it to include all of the above? The rest looks fine to me. Once you update it, I should be able to commit this.
          Hide
          Varun Thacker added a comment -

          Thanks Anshum Gupta for the review!

          Updated patch.

          • Old patch had getProps and setProps in SplitShard. Renamed it to getProperties and setProperties to make it uniform
          • We don't have a specific call for MIGRATE and ADDREPLICAPROP in SolrJ so I have skipped those.
          • Added Properties param to CREATESHARD, ADDREPLICA,
          Show
          Varun Thacker added a comment - Thanks Anshum Gupta for the review! Updated patch. Old patch had getProps and setProps in SplitShard. Renamed it to getProperties and setProperties to make it uniform We don't have a specific call for MIGRATE and ADDREPLICAPROP in SolrJ so I have skipped those. Added Properties param to CREATESHARD, ADDREPLICA,
          Hide
          ASF subversion and git services added a comment -

          Commit 1646350 from Anshum Gupta in branch 'dev/trunk'
          [ https://svn.apache.org/r1646350 ]

          SOLR-5922: Add support for adding core properties to SolrJ Collection Admin Request calls

          Show
          ASF subversion and git services added a comment - Commit 1646350 from Anshum Gupta in branch 'dev/trunk' [ https://svn.apache.org/r1646350 ] SOLR-5922 : Add support for adding core properties to SolrJ Collection Admin Request calls
          Hide
          ASF subversion and git services added a comment -

          Commit 1646352 from Anshum Gupta in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1646352 ]

          SOLR-5922: Add support for adding core properties to SolrJ Collection Admin Request calls (merge from trunk)

          Show
          ASF subversion and git services added a comment - Commit 1646352 from Anshum Gupta in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1646352 ] SOLR-5922 : Add support for adding core properties to SolrJ Collection Admin Request calls (merge from trunk)
          Hide
          Anshum Gupta added a comment -

          Thanks Varun!

          Show
          Anshum Gupta added a comment - Thanks Varun!
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Anshum Gupta
              Reporter:
              Varun Thacker
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development