Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9038

Support snapshot management functionality for a solr collection

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      Currently work is under-way to implement backup/restore API for Solr cloud (SOLR-5750). SOLR-5750 is about providing an ability to "copy" index files and collection metadata to a configurable location.

      In addition to this, we should also provide a facility to create "named" snapshots for Solr collection. Here by "snapshot" I mean configuring the underlying Lucene IndexDeletionPolicy to not delete a specific commit point (e.g. using PersistentSnapshotIndexDeletionPolicy). This should not be confused with SOLR-5340 which implements core level "backup" functionality.

      The primary motivation of this feature is to decouple recording/preserving a known consistent state of a collection from actually "copying" the relevant files to a physically separate location. This decoupling have number of advantages

      • We can use specialized data-copying tools for transferring Solr index files. e.g. in Hadoop environment, typically distcp tool is used to copy files from one location to other. This tool provides various options to configure degree of parallelism, bandwidth usage as well as integration with different types and versions of file systems (e.g. AWS S3, Azure Blob store etc.)
      • This separation of concern would also help Solr to focus on the key functionality (i.e. querying and indexing) while delegating the copy operation to the tools built for that purpose.
      • Users can decide if/when to copy the data files as against creating a snapshot. e.g. a user may want to create a snapshot of a collection before making an experimental change (e.g. updating/deleting docs, schema change etc.). If the experiment is successful, he can delete the snapshot (without having to copy the files). If the experiment is failed, then he can copy the files associated with the snapshot and restore.

      Note that Apache Blur project is also providing a similar feature BLUR-132

        Issue Links

          Activity

          Hide
          dsmiley David Smiley added a comment -

          Cool. So I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit? And in some separate issue, a rollback ability, I presume.

          Show
          dsmiley David Smiley added a comment - Cool. So I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit? And in some separate issue, a rollback ability, I presume.
          Hide
          dsmiley David Smiley added a comment -

          Perhaps another way to view this feature proposed here is to have a commit optionally include a persistent name (or variable name-value metadata for that matter) that will be included with the IndexCommit that is persisted. That would be a somewhat simple way to think of this feature, and needn't involve any SolrCloud related stuff. Of course this data would need to flow-through in all the places commit boolean does, which is a lot of places, but I don't think it would be hard/complicated.

          A separate issue might be a custom deletion policy that treats commits with certain special name-value pairs specially, like keeping them around forever and letting another API/process expressly delete them when asked. Perhaps only the latest commit that has a certain "name" is kept if there is an older one by the same name.

          Show
          dsmiley David Smiley added a comment - Perhaps another way to view this feature proposed here is to have a commit optionally include a persistent name (or variable name-value metadata for that matter) that will be included with the IndexCommit that is persisted. That would be a somewhat simple way to think of this feature, and needn't involve any SolrCloud related stuff. Of course this data would need to flow-through in all the places commit boolean does, which is a lot of places, but I don't think it would be hard/complicated. A separate issue might be a custom deletion policy that treats commits with certain special name-value pairs specially, like keeping them around forever and letting another API/process expressly delete them when asked. Perhaps only the latest commit that has a certain "name" is kept if there is an older one by the same name.
          Hide
          dsmiley David Smiley added a comment -

          Committing with metadata: one thing that occurred to me is that currently a commit is short-circuited if there is no new data. But a commit with metadata needs be persisted (unless the metadata is identical).

          Show
          dsmiley David Smiley added a comment - Committing with metadata: one thing that occurred to me is that currently a commit is short-circuited if there is no new data. But a commit with metadata needs be persisted (unless the metadata is identical).
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Hi David Smiley thanks for the comments

          >>I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit?

          Yes that is correct.

          >>And in some separate issue, a rollback ability, I presume.

          I am thinking to use "restore" capability for this (SOLR-5750). The idea here is that if the "snapshot" needs to restored, it should be exported to a separate location (Exported snapshot is equivalent to a backup). Since "rollback" would be less frequent than snapshot "creation", it should be acceptable to use the "restore" work-flow even if it is less efficient for simplicity and uniformity. But we can always revisit this if there are use-cases.

          >>Perhaps another way to view this feature proposed here is to have a commit optionally include a persistent name (or variable name-value metadata for that matter) that will be included with the IndexCommit that is persisted. That would be a somewhat simple way to think of this feature, and needn't involve any SolrCloud related stuff. Of course this data would need to flow-through in all the places commit boolean does, which is a lot of places, but I don't think it would be hard/complicated.

          I am thinking to define new APIs at collection and core level (CREATESNAPSHOT/DELETESNAPSHOT/LISTSNAPSHOTS). The collection level "CREATESNAPSHOT" operation would be implemented in the Overseer (just like BACKUP/RESTORE). The only difference is that it would invoke core level "CREATESNAPSHOT" API for each of the shard leader replica (instead of BACKUP API). It will also copy the ZK configuration at the specified location.

          Once the snapshot is created for an index commit, the corresponding files will be available for download. This download can be implemented without going through the Overseer. e.g.
          -> If Solr is running on a Hadoop/HDFS cluster, we can use distcp tool to copy the files.
          -> We can use replication handler functionality to copy the files (This can be wrapped as a Solr API or a command line tool).

          I am not quite sure if we utilize the "commit" workflow for snapshot creation, how would we capture the collection metadata?

          Show
          hgadre Hrishikesh Gadre added a comment - Hi David Smiley thanks for the comments >>I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit? Yes that is correct. >>And in some separate issue, a rollback ability, I presume. I am thinking to use "restore" capability for this ( SOLR-5750 ). The idea here is that if the "snapshot" needs to restored, it should be exported to a separate location (Exported snapshot is equivalent to a backup). Since "rollback" would be less frequent than snapshot "creation", it should be acceptable to use the "restore" work-flow even if it is less efficient for simplicity and uniformity. But we can always revisit this if there are use-cases. >>Perhaps another way to view this feature proposed here is to have a commit optionally include a persistent name (or variable name-value metadata for that matter) that will be included with the IndexCommit that is persisted. That would be a somewhat simple way to think of this feature, and needn't involve any SolrCloud related stuff. Of course this data would need to flow-through in all the places commit boolean does, which is a lot of places, but I don't think it would be hard/complicated. I am thinking to define new APIs at collection and core level (CREATESNAPSHOT/DELETESNAPSHOT/LISTSNAPSHOTS). The collection level "CREATESNAPSHOT" operation would be implemented in the Overseer (just like BACKUP/RESTORE). The only difference is that it would invoke core level "CREATESNAPSHOT" API for each of the shard leader replica (instead of BACKUP API). It will also copy the ZK configuration at the specified location. Once the snapshot is created for an index commit, the corresponding files will be available for download. This download can be implemented without going through the Overseer. e.g. -> If Solr is running on a Hadoop/HDFS cluster, we can use distcp tool to copy the files. -> We can use replication handler functionality to copy the files (This can be wrapped as a Solr API or a command line tool). I am not quite sure if we utilize the "commit" workflow for snapshot creation, how would we capture the collection metadata?
          Hide
          dsmiley David Smiley added a comment -

          >>I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit?

          Yes that is correct.

          Yet I'm now unsure we're actually talking about the same thing, given everything else you've said. If this issue you propose is anything more than adding commit metadata (copying segments to another place is more than just adding commit metadata), then how is this issue different then SOLR-5750? I understand we want to leverage storage level efficiencies (e.g. distcp) but this issue doesn't seem to actually be about that. Its appearing to be redundant with SOLR-5750. Or perhaps you mean, extend/enhance the result of SOLR-5750 so that we have an API to list & delete the backups without requiring a client to go to the shared file system to observe what backups there are? Big +1 to that and if so please clarify the title/description and add a requires linkage to SOLR-5750. Again if you mean that, then mentioning HDFS etc. is a distraction to this issue's purpose.

          Show
          dsmiley David Smiley added a comment - >>I presume by "snapshot", we're talking about named (or numbered) Lucene IndexCommit objects across all replicas of a Solr Collection? And then, in SOLR-5750 or future patch, the "backup" capability might optionally make reference to a named snapshot instead of just taking the last IndexCommit? Yes that is correct. Yet I'm now unsure we're actually talking about the same thing, given everything else you've said. If this issue you propose is anything more than adding commit metadata (copying segments to another place is more than just adding commit metadata), then how is this issue different then SOLR-5750 ? I understand we want to leverage storage level efficiencies (e.g. distcp) but this issue doesn't seem to actually be about that. Its appearing to be redundant with SOLR-5750 . Or perhaps you mean, extend/enhance the result of SOLR-5750 so that we have an API to list & delete the backups without requiring a client to go to the shared file system to observe what backups there are? Big +1 to that and if so please clarify the title/description and add a requires linkage to SOLR-5750 . Again if you mean that, then mentioning HDFS etc. is a distraction to this issue's purpose.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Sorry for the confusion. Yes it make sense to defer any "copying" behavior to SOLR-5750. We can customize "backup functionality" to incorporate this (e.g. ability to back up previously created snapshot). But from the perspective of this JIRA, let's focus on indexed data only.

          As you mentioned in your earlier comments, we can use the "commit" workflow to create a named snapshot. But we still need a way to list the previously created snapshots and an ability to delete the snapshots. Here the "delete snapshot" functionality can just remove the corresponding index commit metadata. This way during subsequent index merge, Lucene can perform the cleanup.

          Does that make sense? If yes, I do have following questions

          • How would the "list snapshots" and "delete snapshot" APIs look like? Do we need to provide them just at the core level or at the collection level as well?
          • Would we allow "destructive" operations (e.g. delete replica/shard) when we have one or more snapshots?
          • It seems to me that the "commit" request will be executed by all replicas for a given collection. What should happen when a "commit" request can not be processed by a replica (since it may be down) ? We may need to ensure that during the replica "recovery" it also fetches the information about commit metadata.
          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Sorry for the confusion. Yes it make sense to defer any "copying" behavior to SOLR-5750 . We can customize "backup functionality" to incorporate this (e.g. ability to back up previously created snapshot). But from the perspective of this JIRA, let's focus on indexed data only. As you mentioned in your earlier comments, we can use the "commit" workflow to create a named snapshot. But we still need a way to list the previously created snapshots and an ability to delete the snapshots. Here the "delete snapshot" functionality can just remove the corresponding index commit metadata. This way during subsequent index merge, Lucene can perform the cleanup. Does that make sense? If yes, I do have following questions How would the "list snapshots" and "delete snapshot" APIs look like? Do we need to provide them just at the core level or at the collection level as well? Would we allow "destructive" operations (e.g. delete replica/shard) when we have one or more snapshots? It seems to me that the "commit" request will be executed by all replicas for a given collection. What should happen when a "commit" request can not be processed by a replica (since it may be down) ? We may need to ensure that during the replica "recovery" it also fetches the information about commit metadata.
          Hide
          dsmiley David Smiley added a comment -

          I think we may be understanding each other again. We might not want to call this a snapshot simply because there are remnants of that naming within replication & backup (e.g. "Snapshooter"). Instead I propose naming it closer to what it actually is implemented as – like a "commit lease" or "snapshot commit" (the operative word being "commit"). Of course others may want to comment; I have no conviction. For now lets continue with "snapshot commit" as it retains both words, and is a decent name, I think.

          As you mentioned in your earlier comments, we can use the "commit" workflow to create a named snapshot.

          Does that make sense?

          Yes!

          How would the "list snapshots" and "delete snapshot" APIs look like? Do we need to provide them just at the core level or at the collection level as well?

          I think the data to be listed is fundamentally at the core, so certainly the core level. But a collection level API is needed – it could simply take the distinct union list from asking each leader. It could list the snapshot commits not common to all in a separate list, if there's any utility in that?

          Would we allow "destructive" operations (e.g. delete replica/shard) when we have one or more snapshots?

          I think so. Not doing so might be a pain, and it's not evident to me it's important to worry about it.

          It seems to me that the "commit" request will be executed by all replicas for a given collection. What should happen when a "commit" request can not be processed by a replica (since it may be down) ? We may need to ensure that during the replica "recovery" it also fetches the information about commit metadata.

          Hmm; good point. That might be a PITA unfortunately. Perhaps a snapshot commit needs to block for all replicas to not be in recovery first? That seems much easier than trying to get replicas in recovery to somehow get IndexCommit data which I think is kinda impossible / infeasible. However, another bad situation is when there are already successful snapshot commits, and then for whatever reason a replica goes into recovery – full recovery, and thus only grabs the latest commit (which might not even be a snapshot commit. So perhaps recovering replicas need to ask to replicate not just the latest commit but all snapshot commits as well. Seems pretty doable. One would hope that the commits would share lots of big segments, but they might not. I don't think this scenario would block an initial release. Possible but too bad.

          Show
          dsmiley David Smiley added a comment - I think we may be understanding each other again. We might not want to call this a snapshot simply because there are remnants of that naming within replication & backup (e.g. "Snapshooter"). Instead I propose naming it closer to what it actually is implemented as – like a "commit lease" or "snapshot commit" (the operative word being "commit"). Of course others may want to comment; I have no conviction. For now lets continue with "snapshot commit" as it retains both words, and is a decent name, I think. As you mentioned in your earlier comments, we can use the "commit" workflow to create a named snapshot. Does that make sense? Yes! How would the "list snapshots" and "delete snapshot" APIs look like? Do we need to provide them just at the core level or at the collection level as well? I think the data to be listed is fundamentally at the core, so certainly the core level. But a collection level API is needed – it could simply take the distinct union list from asking each leader. It could list the snapshot commits not common to all in a separate list, if there's any utility in that? Would we allow "destructive" operations (e.g. delete replica/shard) when we have one or more snapshots? I think so. Not doing so might be a pain, and it's not evident to me it's important to worry about it. It seems to me that the "commit" request will be executed by all replicas for a given collection. What should happen when a "commit" request can not be processed by a replica (since it may be down) ? We may need to ensure that during the replica "recovery" it also fetches the information about commit metadata. Hmm; good point. That might be a PITA unfortunately. Perhaps a snapshot commit needs to block for all replicas to not be in recovery first? That seems much easier than trying to get replicas in recovery to somehow get IndexCommit data which I think is kinda impossible / infeasible. However, another bad situation is when there are already successful snapshot commits, and then for whatever reason a replica goes into recovery – full recovery, and thus only grabs the latest commit (which might not even be a snapshot commit. So perhaps recovering replicas need to ask to replicate not just the latest commit but all snapshot commits as well. Seems pretty doable. One would hope that the commits would share lots of big segments, but they might not. I don't think this scenario would block an initial release. Possible but too bad.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Thanks for the comments

          >>I think so. Not doing so might be a pain, and it's not evident to me it's important to worry about it.

          If we are going to allow the "backup" operation to use this snapshot commit in future, then I think we need to make sure that that snapshot commit is preserved during collection configuration changes. If the snapshot commit is created on all replicas for a shard, then it probably is OK to delete one or more replicas. But I am not sure how would we handle the case when a shard containing a one or more snapshot commits is deleted.

          >>Perhaps a snapshot commit needs to block for all replicas to not be in recovery first? That seems much easier than trying to get replicas in recovery to somehow get IndexCommit data which I think is kinda impossible / infeasible. However, another bad situation is when there are already successful snapshot commits, and then for whatever reason a replica goes into recovery – full recovery, and thus only grabs the latest commit (which might not even be a snapshot commit. So perhaps recovering replicas need to ask to replicate not just the latest commit but all snapshot commits as well. Seems pretty doable. One would hope that the commits would share lots of big segments, but they might not. I don't think this scenario would block an initial release. Possible but too bad.

          I agree that requiring replicas to transfer snapshot commits during recovery may not be a good idea since in case of large collections it will increase the size of data transferred over the network. I am also not very sure if we should block for all replicas to be "active" before creating a snapshot since on a large cluster it would more likely that one or more replicas would be "down" or "recovering".

          I do have an alternative design in mind, but just want to make sure that we are on the same page regarding overall semantics before diving into details

          Thoughts?

          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Thanks for the comments >>I think so. Not doing so might be a pain, and it's not evident to me it's important to worry about it. If we are going to allow the "backup" operation to use this snapshot commit in future, then I think we need to make sure that that snapshot commit is preserved during collection configuration changes. If the snapshot commit is created on all replicas for a shard, then it probably is OK to delete one or more replicas. But I am not sure how would we handle the case when a shard containing a one or more snapshot commits is deleted. >>Perhaps a snapshot commit needs to block for all replicas to not be in recovery first? That seems much easier than trying to get replicas in recovery to somehow get IndexCommit data which I think is kinda impossible / infeasible. However, another bad situation is when there are already successful snapshot commits, and then for whatever reason a replica goes into recovery – full recovery, and thus only grabs the latest commit (which might not even be a snapshot commit. So perhaps recovering replicas need to ask to replicate not just the latest commit but all snapshot commits as well. Seems pretty doable. One would hope that the commits would share lots of big segments, but they might not. I don't think this scenario would block an initial release. Possible but too bad. I agree that requiring replicas to transfer snapshot commits during recovery may not be a good idea since in case of large collections it will increase the size of data transferred over the network. I am also not very sure if we should block for all replicas to be "active" before creating a snapshot since on a large cluster it would more likely that one or more replicas would be "down" or "recovering". I do have an alternative design in mind, but just want to make sure that we are on the same page regarding overall semantics before diving into details Thoughts?
          Hide
          dsmiley David Smiley added a comment -

          If we are going to allow the "backup" operation to use this snapshot commit in future, then I think we need to make sure that that snapshot commit is preserved during collection configuration changes. If the snapshot commit is created on all replicas for a shard, then it probably is OK to delete one or more replicas. But I am not sure how would we handle the case when a shard containing a one or more snapshot commits is deleted.

          There's no issue, I think, if a replica is deleted. If a whole shard is deleted, then I think it's okay too – it won't be backed up – there's nothing left

          I agree that requiring replicas to transfer snapshot commits during recovery may not be a good idea since in case of large collections it will increase the size of data transferred over the network.

          I don't think it's a blocker to the approach... it's just the price one pays to recover in the presence of snapshot commits. Other improvements around how Lucene segments merge might make more sense to optimize this such that segments can only be merged if the IndexCommits pointing to them are consistent. If this idea were implemented, and If one were to do an optimize (as a hypothetical example to explain the effect), they would have a segment for each snapshot commit, with disjoint documents (no duplication). Pretty good, I think. But this would clearly be it's own issue

          Show
          dsmiley David Smiley added a comment - If we are going to allow the "backup" operation to use this snapshot commit in future, then I think we need to make sure that that snapshot commit is preserved during collection configuration changes. If the snapshot commit is created on all replicas for a shard, then it probably is OK to delete one or more replicas. But I am not sure how would we handle the case when a shard containing a one or more snapshot commits is deleted. There's no issue, I think, if a replica is deleted. If a whole shard is deleted, then I think it's okay too – it won't be backed up – there's nothing left I agree that requiring replicas to transfer snapshot commits during recovery may not be a good idea since in case of large collections it will increase the size of data transferred over the network. I don't think it's a blocker to the approach... it's just the price one pays to recover in the presence of snapshot commits. Other improvements around how Lucene segments merge might make more sense to optimize this such that segments can only be merged if the IndexCommits pointing to them are consistent. If this idea were implemented, and If one were to do an optimize (as a hypothetical example to explain the effect), they would have a segment for each snapshot commit, with disjoint documents (no duplication). Pretty good, I think. But this would clearly be it's own issue
          Hide
          hgadre Hrishikesh Gadre added a comment - - edited

          David Smiley It seems like we shouldn't override commit work-flow with snapshot creation since SOLR-6761 allows administrators to disable explicit commit requests. I think defining a CREATESNAPSHOT request make more sense.

          Show
          hgadre Hrishikesh Gadre added a comment - - edited David Smiley It seems like we shouldn't override commit work-flow with snapshot creation since SOLR-6761 allows administrators to disable explicit commit requests. I think defining a CREATESNAPSHOT request make more sense.
          Hide
          dsmiley David Smiley added a comment -

          I definitely wouldn't let the existence of the IgnoreCommitOptimizeUpdateProcessorFactory alter what happens in this issue. I've only seen it used once to prevent a bad Solr client (pysolr I think). That's not to say I disagree with a hypothetical CREATESNAPSHOT... but at least having this be an option to a commit makes it clear that a snapshot is tightly associated with a commit – it is. And perhaps it could work with commitWithin if you aren't too particular about the precise timing of when the snapshot is created.

          Show
          dsmiley David Smiley added a comment - I definitely wouldn't let the existence of the IgnoreCommitOptimizeUpdateProcessorFactory alter what happens in this issue. I've only seen it used once to prevent a bad Solr client (pysolr I think). That's not to say I disagree with a hypothetical CREATESNAPSHOT... but at least having this be an option to a commit makes it clear that a snapshot is tightly associated with a commit – it is. And perhaps it could work with commitWithin if you aren't too particular about the precise timing of when the snapshot is created.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Thanks for the feedback.

          The reason why I brought this up to make sure that this feature works nicely with the existing functionality (and requirements). The good point about integrating snapshot creation with commit work-flow is that the snapshot would always have the latest committed index state.

          Reviewing the current backup implementation, it looks like the backup logic does not commit anything. It just copies the latest commit to the backup directory. Obviously user can manually issue a hard commit before invoking backup operation or let the backup be created with the latest auto hard committed data. From Solr perspective these two are distinct operations. I wonder if there is a specific reason for this?

          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Thanks for the feedback. The reason why I brought this up to make sure that this feature works nicely with the existing functionality (and requirements). The good point about integrating snapshot creation with commit work-flow is that the snapshot would always have the latest committed index state. Reviewing the current backup implementation, it looks like the backup logic does not commit anything. It just copies the latest commit to the backup directory. Obviously user can manually issue a hard commit before invoking backup operation or let the backup be created with the latest auto hard committed data. From Solr perspective these two are distinct operations. I wonder if there is a specific reason for this?
          Hide
          dsmiley David Smiley added a comment -

          For clarification, do you mean you wonder why committing is distinct from doing a backup? Well I guess that adds flexibility – you might not want to incur opening of searchers just to do a regular backup. I suppose then perhaps someone likewise might want to do a snapshot without a commit using the same rationale? Although if snapshots are implemented using Lucene commit metadata (which sounds ideal to me), then we have to issue a commit but it could be done without opening a searcher. In that case it may or may not accompany an explicit commit.

          Show
          dsmiley David Smiley added a comment - For clarification, do you mean you wonder why committing is distinct from doing a backup? Well I guess that adds flexibility – you might not want to incur opening of searchers just to do a regular backup. I suppose then perhaps someone likewise might want to do a snapshot without a commit using the same rationale? Although if snapshots are implemented using Lucene commit metadata (which sounds ideal to me), then we have to issue a commit but it could be done without opening a searcher. In that case it may or may not accompany an explicit commit.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Thanks for the comments. I have started working on this feature. Here is the first cut implementation,

          https://github.com/hgadre/lucene-solr/tree/SOLR-9038

          • It includes snapshot management only at the core level. For now I have defined a new API (CREATESNAPSHOT) primarily to simplify initial implementation and get an end-to-end functionality in place.
          • I am working on implementing collection level snapshot management functionality.

          Please take a look and let me have your feedback.

          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Thanks for the comments. I have started working on this feature. Here is the first cut implementation, https://github.com/hgadre/lucene-solr/tree/SOLR-9038 It includes snapshot management only at the core level. For now I have defined a new API (CREATESNAPSHOT) primarily to simplify initial implementation and get an end-to-end functionality in place. I am working on implementing collection level snapshot management functionality. Please take a look and let me have your feedback.
          Hide
          dsmiley David Smiley added a comment -

          I look forward to checking this out but FYI it may take a day or two for me to get back to this as I'm on vacation.

          Show
          dsmiley David Smiley added a comment - I look forward to checking this out but FYI it may take a day or two for me to get back to this as I'm on vacation.
          Hide
          dsmiley David Smiley added a comment -

          BTW i've been adding comments at the GH commits on Hrishikesh Gadre's branch there. Overall it's looking good – I like it. My only concern (repeating myself from GH) is that SolrPersistentSnapshotManager is a bolt-on to Solr's IndexDeletionPolicyWrapper when perhaps it should be integrated (one cohesive whole)? Or keep it bolt-on but make the code that's in IDPW a separate bolt-on as well? It's debatable... another opinion would be nice.

          BTW IMO "SolrDelectionPolicy" would be a better name to IndexDeletionPolicyWrapper.

          Show
          dsmiley David Smiley added a comment - BTW i've been adding comments at the GH commits on Hrishikesh Gadre 's branch there. Overall it's looking good – I like it. My only concern (repeating myself from GH) is that SolrPersistentSnapshotManager is a bolt-on to Solr's IndexDeletionPolicyWrapper when perhaps it should be integrated (one cohesive whole)? Or keep it bolt-on but make the code that's in IDPW a separate bolt-on as well? It's debatable... another opinion would be nice. BTW IMO "SolrDelectionPolicy" would be a better name to IndexDeletionPolicyWrapper.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Thanks for the comments.

          My only concern (repeating myself from GH) is that SolrPersistentSnapshotManager is a bolt-on to Solr's IndexDeletionPolicyWrapper when perhaps it should be integrated (one cohesive whole)? Or keep it bolt-on but make the code that's in IDPW a separate bolt-on as well? It's debatable... another opinion would be nice.

          I am not quite sure what you are alluding to. But here is my thinking.

          SolrPersistentSnapshotManager -> takes care of persisting/querying the snapshot meta-data
          IndexDeletionPolicyWrapper -> takes care of preserving the index commits for the configured snapshots.

          This keeps the code modular and easy to understand (as against adding all the logic in IndexDeletionPolicyWrapper directly). Previously SolrPersistentSnapshotManager was accessible only via IndexDeletionPolicyWrapper. Now I changed the logic to expose SolrPersistentSnapshotManager directly via SolrCore. This allows separation of concerns as described above. Please take a look,
          https://github.com/hgadre/lucene-solr/commit/68c2784b827ae27a002f0de6dfd01d2c9d3b07be

          Does this address your concern?

          May be we can rename SolrPersistentSnapshotManager to SolrSnapshotMetaDataManager for clarity.

          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Thanks for the comments. My only concern (repeating myself from GH) is that SolrPersistentSnapshotManager is a bolt-on to Solr's IndexDeletionPolicyWrapper when perhaps it should be integrated (one cohesive whole)? Or keep it bolt-on but make the code that's in IDPW a separate bolt-on as well? It's debatable... another opinion would be nice. I am not quite sure what you are alluding to. But here is my thinking. SolrPersistentSnapshotManager -> takes care of persisting/querying the snapshot meta-data IndexDeletionPolicyWrapper -> takes care of preserving the index commits for the configured snapshots. This keeps the code modular and easy to understand (as against adding all the logic in IndexDeletionPolicyWrapper directly). Previously SolrPersistentSnapshotManager was accessible only via IndexDeletionPolicyWrapper. Now I changed the logic to expose SolrPersistentSnapshotManager directly via SolrCore. This allows separation of concerns as described above. Please take a look, https://github.com/hgadre/lucene-solr/commit/68c2784b827ae27a002f0de6dfd01d2c9d3b07be Does this address your concern? May be we can rename SolrPersistentSnapshotManager to SolrSnapshotMetaDataManager for clarity.
          Hide
          dsmiley David Smiley added a comment -

          Previously SolrPersistentSnapshotManager was accessible only via IndexDeletionPolicyWrapper. Now I changed the logic to expose SolrPersistentSnapshotManager directly via SolrCore.

          +1 (I reviewed the commit). I think it's a small improvement in the sense that it may be non-obvious to some people that these classes are related.

          It didn't quite get at my point, but I think esp. with a previous improvement that I no longer wish to raise any concern about the relationship between them.

          So perhaps this is ready to commit, though there were some minor improvements I suggested RE Java 8 streams. Do tests pass & "ant precommit"? I can do this July 5th – when I'm back from vacation. Possibly sooner.

          Show
          dsmiley David Smiley added a comment - Previously SolrPersistentSnapshotManager was accessible only via IndexDeletionPolicyWrapper. Now I changed the logic to expose SolrPersistentSnapshotManager directly via SolrCore. +1 (I reviewed the commit). I think it's a small improvement in the sense that it may be non-obvious to some people that these classes are related. It didn't quite get at my point, but I think esp. with a previous improvement that I no longer wish to raise any concern about the relationship between them. So perhaps this is ready to commit, though there were some minor improvements I suggested RE Java 8 streams. Do tests pass & "ant precommit"? I can do this July 5th – when I'm back from vacation. Possibly sooner.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          David Smiley Thanks for the feedback!

          I have created a new sub-task to track the implementation of this functionality at Solr core level (which is now complete). I am planning to work on extending it at collection level early next week.

          , though there were some minor improvements I suggested RE Java 8 streams.

          I think I have addressed those comments. Please take a look at the attached patch for SOLR-9269

          Do tests pass & "ant precommit"?

          Yes I verified that all tests as well as precommit is passing.

          Show
          hgadre Hrishikesh Gadre added a comment - David Smiley Thanks for the feedback! I have created a new sub-task to track the implementation of this functionality at Solr core level (which is now complete). I am planning to work on extending it at collection level early next week. , though there were some minor improvements I suggested RE Java 8 streams. I think I have addressed those comments. Please take a look at the attached patch for SOLR-9269 Do tests pass & "ant precommit"? Yes I verified that all tests as well as precommit is passing.
          Hide
          varunthacker Varun Thacker added a comment -

          Hi David,Hrishikesh

          I was trying to document this feature in the ref guide and had a doubt:

          1. If we create a snapshot , there is no way to restore a snapshot directly into the same index? We have to first backup the snapshot and then restore that?

          Show
          varunthacker Varun Thacker added a comment - Hi David,Hrishikesh I was trying to document this feature in the ref guide and had a doubt: 1. If we create a snapshot , there is no way to restore a snapshot directly into the same index? We have to first backup the snapshot and then restore that?
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Varun Thacker Yes currently we don't support restoring a snapshot directly into the same index although it would be a nice feature to add

          I need to update the patch for SOLR-9326. Let me do that in a day or so.

          Show
          hgadre Hrishikesh Gadre added a comment - Varun Thacker Yes currently we don't support restoring a snapshot directly into the same index although it would be a nice feature to add I need to update the patch for SOLR-9326 . Let me do that in a day or so.
          Hide
          varunthacker Varun Thacker added a comment -

          Yes currently we don't support restoring a snapshot directly into the same index although it would be a nice feature to add

          +1 . I think that will bring out the best of the snapshot API . The user simply creates a snapshot , no index files need to be copied . He can then simple restore a snapshot and it will be faster than the backup/restore process

          Show
          varunthacker Varun Thacker added a comment - Yes currently we don't support restoring a snapshot directly into the same index although it would be a nice feature to add +1 . I think that will bring out the best of the snapshot API . The user simply creates a snapshot , no index files need to be copied . He can then simple restore a snapshot and it will be faster than the backup/restore process
          Hide
          varunthacker Varun Thacker added a comment -

          Also feel free to comment on https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups where I've tried documenting the feature.

          Show
          varunthacker Varun Thacker added a comment - Also feel free to comment on https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups where I've tried documenting the feature.
          Hide
          hgadre Hrishikesh Gadre added a comment -

          Varun Thacker I think we need more details around snapshot management functionality. I have a write-up on that. Should I submit the text to this JIRA ?

          Show
          hgadre Hrishikesh Gadre added a comment - Varun Thacker I think we need more details around snapshot management functionality. I have a write-up on that. Should I submit the text to this JIRA ?
          Hide
          varunthacker Varun Thacker added a comment -

          Hi Hrishikesh,

          Here is fine. I'll move it over to the ref guide once you have posted it here. Thanks!

          Show
          varunthacker Varun Thacker added a comment - Hi Hrishikesh, Here is fine. I'll move it over to the ref guide once you have posted it here. Thanks!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1381dd9287a23c950eaaa3c258249a5ebc812f35 in lucene-solr's branch refs/heads/master from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1381dd9 ]

          SOLR-9055: Make collection backup/restore extensible.

          • Introduced a parameter for the Backup operation to specify index backup strategy.
          • Introduced two strategies for backing up index data.
          • One using core Admin API (BACKUPCORE)
          • Other skipping the backup of index data altogether. This is useful when
            the index data is copied via an external mechanism in combination with named
            snapshots (Please refer to SOLR-9038 for details)
          • In future we can add additional implementations of this interface (e.g. based on HDFS snapshots etc.)
          • Added a backup property to record the Solr version. This helps to check the compatibility
            of backup with respect to the current version during the restore operation. This
            compatibility check is not added since its unclear what the Solr level compatibility guidelines
            are. But at-least having version information as part of the backup would be very useful.
          Show
          jira-bot ASF subversion and git services added a comment - Commit 1381dd9287a23c950eaaa3c258249a5ebc812f35 in lucene-solr's branch refs/heads/master from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1381dd9 ] SOLR-9055 : Make collection backup/restore extensible. Introduced a parameter for the Backup operation to specify index backup strategy. Introduced two strategies for backing up index data. One using core Admin API (BACKUPCORE) Other skipping the backup of index data altogether. This is useful when the index data is copied via an external mechanism in combination with named snapshots (Please refer to SOLR-9038 for details) In future we can add additional implementations of this interface (e.g. based on HDFS snapshots etc.) Added a backup property to record the Solr version. This helps to check the compatibility of backup with respect to the current version during the restore operation. This compatibility check is not added since its unclear what the Solr level compatibility guidelines are. But at-least having version information as part of the backup would be very useful.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 03cac8c7b5cb03a0940b1810bcece58466744f26 in lucene-solr's branch refs/heads/branch_6x from markrmiller
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=03cac8c ]

          SOLR-9055: Make collection backup/restore extensible.

          • Introduced a parameter for the Backup operation to specify index backup strategy.
          • Introduced two strategies for backing up index data.
          • One using core Admin API (BACKUPCORE)
          • Other skipping the backup of index data altogether. This is useful when
            the index data is copied via an external mechanism in combination with named
            snapshots (Please refer to SOLR-9038 for details)
          • In future we can add additional implementations of this interface (e.g. based on HDFS snapshots etc.)
          • Added a backup property to record the Solr version. This helps to check the compatibility
            of backup with respect to the current version during the restore operation. This
            compatibility check is not added since its unclear what the Solr level compatibility guidelines
            are. But at-least having version information as part of the backup would be very useful.
          1. Conflicts:
          2. solr/CHANGES.txt
          Show
          jira-bot ASF subversion and git services added a comment - Commit 03cac8c7b5cb03a0940b1810bcece58466744f26 in lucene-solr's branch refs/heads/branch_6x from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=03cac8c ] SOLR-9055 : Make collection backup/restore extensible. Introduced a parameter for the Backup operation to specify index backup strategy. Introduced two strategies for backing up index data. One using core Admin API (BACKUPCORE) Other skipping the backup of index data altogether. This is useful when the index data is copied via an external mechanism in combination with named snapshots (Please refer to SOLR-9038 for details) In future we can add additional implementations of this interface (e.g. based on HDFS snapshots etc.) Added a backup property to record the Solr version. This helps to check the compatibility of backup with respect to the current version during the restore operation. This compatibility check is not added since its unclear what the Solr level compatibility guidelines are. But at-least having version information as part of the backup would be very useful. Conflicts: solr/CHANGES.txt
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit b02e7a902e80c82750f8e8c403b3a9c5e887c43b in lucene-solr's branch refs/heads/master from Yonik Seeley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b02e7a9 ]

          SOLR-9038: add snapshotscli.sh and supporting code

          Show
          jira-bot ASF subversion and git services added a comment - Commit b02e7a902e80c82750f8e8c403b3a9c5e887c43b in lucene-solr's branch refs/heads/master from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b02e7a9 ] SOLR-9038 : add snapshotscli.sh and supporting code
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 46f5f45862a7e7710c0acc8bcb72547da19739b3 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=46f5f45 ]

          SOLR-9038: add snapshotscli.sh and supporting code

          Show
          jira-bot ASF subversion and git services added a comment - Commit 46f5f45862a7e7710c0acc8bcb72547da19739b3 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=46f5f45 ] SOLR-9038 : add snapshotscli.sh and supporting code
          Hide
          mkhludnev Mikhail Khludnev added a comment -

          Isn't it done and can be resolved?

          Show
          mkhludnev Mikhail Khludnev added a comment - Isn't it done and can be resolved?

            People

            • Assignee:
              dsmiley David Smiley
              Reporter:
              hgadre Hrishikesh Gadre
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development