Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2, 5.0
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      We should bring back the old aliasing feature, but for SolrCloud and with the ability to alias one collection to many.

      The old alias feature was of more limited use and had some problems, so we dropped it, but I think we can do this in a more useful way with SolrCloud, and at a level were it's not invasive to the CoreContainer.

      Initially, the search side will allowing mapping a single alias to multiple collections, but the index side will only support mapping a single alias to a single collection.

      1. CDH-4497.patch
        57 kB
        Mark Miller
      2. SOLR-4497.patch
        51 kB
        Mark Miller

        Issue Links

          Activity

          Hide
          Mark Miller added a comment -

          Here is a patch for an initial pass.

          • There are two new actions for the collections api, createalias and deletealias.
          • createalias is also currently used to update.
          • On the search side you can map from one alias to multiple collections
          • On the update side, only a one to one mapping will work.
          Show
          Mark Miller added a comment - Here is a patch for an initial pass. There are two new actions for the collections api, createalias and deletealias. createalias is also currently used to update. On the search side you can map from one alias to multiple collections On the update side, only a one to one mapping will work.
          Hide
          Mark Miller added a comment -

          I'd like to get this into 4.2, so I'll commit so that it can start baking and hopefully I can get some more feedback.

          Show
          Mark Miller added a comment - I'd like to get this into 4.2, so I'll commit so that it can start baking and hopefully I can get some more feedback.
          Hide
          Yonik Seeley added a comment -

          We should also strive for 100% visibility of an alias once the call returns.
          For example, one should be able to script a call to create an alias and then immediately use it w/o worrying about some nodes not seeing it yet.
          The easiest way to implement this would be to re-read the aliases from ZK on any "miss".

          Show
          Yonik Seeley added a comment - We should also strive for 100% visibility of an alias once the call returns. For example, one should be able to script a call to create an alias and then immediately use it w/o worrying about some nodes not seeing it yet. The easiest way to implement this would be to re-read the aliases from ZK on any "miss".
          Hide
          Mark Miller added a comment -

          Okay, I'll look into that.

          Show
          Mark Miller added a comment - Okay, I'll look into that.
          Hide
          Tim Vaillancourt added a comment -

          What about the case where the alias already exists, but just has a new destination (no alias miss)? Can/should the call return success on the change reaching Zookeeper, or the running alias on all nodes? I'd prefer the later but I expect that may be a significant amount of work.

          Show
          Tim Vaillancourt added a comment - What about the case where the alias already exists, but just has a new destination (no alias miss)? Can/should the call return success on the change reaching Zookeeper, or the running alias on all nodes? I'd prefer the later but I expect that may be a significant amount of work.
          Hide
          Mark Miller added a comment -

          Good point Tim. I imagine I will have to setting for the former initially, but def thinking about how to deal with that case.

          Show
          Mark Miller added a comment - Good point Tim. I imagine I will have to setting for the former initially, but def thinking about how to deal with that case.
          Hide
          Mark Miller added a comment -

          how to deal with that case.

          So, it's not really as solid a solution as I'd like, but I've handled it this way:

          The Overseer that actually writes the new Alias will wait until it locally gets pinged by ZK and gets the latest Aliases. It then waits a small fudge factor for other nodes that may get updated slightly behind and returns.

          Show
          Mark Miller added a comment - how to deal with that case. So, it's not really as solid a solution as I'd like, but I've handled it this way: The Overseer that actually writes the new Alias will wait until it locally gets pinged by ZK and gets the latest Aliases. It then waits a small fudge factor for other nodes that may get updated slightly behind and returns.
          Hide
          Mark Miller added a comment -

          New patch.

          • Additional testing
          • Fixes the case when the node you are hitting does not have a piece of the first collection in your alias list
          • Updates aliases and does a retry on alias miss
          • Tries to wait for alias changes as mentioned in my comment above.
          Show
          Mark Miller added a comment - New patch. Additional testing Fixes the case when the node you are hitting does not have a piece of the first collection in your alias list Updates aliases and does a retry on alias miss Tries to wait for alias changes as mentioned in my comment above.
          Hide
          Mark Miller added a comment -

          I'm ready to commit this soon - I'd like to get it in for 4.2.

          Show
          Mark Miller added a comment - I'm ready to commit this soon - I'd like to get it in for 4.2.
          Hide
          Tim Vaillancourt added a comment - - edited

          I think that is a good start. The change wouldn't be 100% immediate on return of the call, but designing the client/app to expect this could be ok. If there was a STATUS call on the Cores API that could detail the node's running alias, one could write an external script to "check" that the change made it everywhere.

          This lead me to wondering (again without understanding implications and design goals) would it be best to have the Overseer push the change to all nodes (and write to ZK) vs relying only on the change in ZK, which has delays.

          Show
          Tim Vaillancourt added a comment - - edited I think that is a good start. The change wouldn't be 100% immediate on return of the call, but designing the client/app to expect this could be ok. If there was a STATUS call on the Cores API that could detail the node's running alias, one could write an external script to "check" that the change made it everywhere. This lead me to wondering (again without understanding implications and design goals) would it be best to have the Overseer push the change to all nodes (and write to ZK) vs relying only on the change in ZK, which has delays.
          Hide
          Mark Miller added a comment -

          Okay - committing the current state now.

          This lead me to wondering (again without understanding implications and design goals) would it be best to have the Overseer push the change to all nodes (and write to ZK) vs relying only on the change in ZK, which has delays.

          I think this is something we should consider. It actually came up before with the clusterstate.json as well - I think we are actually okay with how that works currently though, but it may make sense to have the overseer to distribute this. Let's make a new JIRA issue and investigate it as a future improvement.

          Show
          Mark Miller added a comment - Okay - committing the current state now. This lead me to wondering (again without understanding implications and design goals) would it be best to have the Overseer push the change to all nodes (and write to ZK) vs relying only on the change in ZK, which has delays. I think this is something we should consider. It actually came up before with the clusterstate.json as well - I think we are actually okay with how that works currently though, but it may make sense to have the overseer to distribute this. Let's make a new JIRA issue and investigate it as a future improvement.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1453687

          SOLR-4497: Collection Aliasing.

          Show
          Commit Tag Bot added a comment - [trunk commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1453687 SOLR-4497 : Collection Aliasing.
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1453691

          SOLR-4497: Collection Aliasing.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1453691 SOLR-4497 : Collection Aliasing.
          Hide
          Uwe Schindler added a comment -

          Closed after release.

          Show
          Uwe Schindler added a comment - Closed after release.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development