Solr
  1. Solr
  2. SOLR-1216

disambiguate the replication command names

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4
    • Component/s: replication (java)
    • Labels:
      None

      Description

      There is a lot of confusion in the naming of various commands such as snappull, snapshot etc. This is a vestige of the script based replication we currently have. The commands can be renamed to make more sense

      • 'snappull' to be renamed to 'sync'
      • 'snapshot' to be renamed to 'backup'

      thoughts?

      1. SOLR-1216.patch
        2 kB
        Noble Paul
      2. SOLR-1216.patch
        3 kB
        Noble Paul
      3. SOLR-1216.patch
        2 kB
        Noble Paul

        Activity

        Hide
        Noble Paul added a comment -
        Show
        Noble Paul added a comment - the relevant mail thread http://markmail.org/thread/zucdwx4p2xmeaejj
        Hide
        Mark Miller added a comment -

        +1.

        I like the names too. Perhaps 'syncFromMaster' ? I could go either way for various reasons.

        Lets also clearup the doc for this stuff as part of the issue. I can work on the wiki part if you'd like.

        Are backups made in the same format as the scripts method (eg can I basically steal that info from the scripts wiki page?)

        Is it possible to replicate a backup or just the live index?

        I also want to put in what happens when you shutdown during a replication - does Solr wait? Abort? If its aborts, are the temp files cleaned up later?

        replicate and polling both to default to on right?

        and the replicate setting just stops any slaves hitting a master with replicate=off from syncing?

        Show
        Mark Miller added a comment - +1. I like the names too. Perhaps 'syncFromMaster' ? I could go either way for various reasons. Lets also clearup the doc for this stuff as part of the issue. I can work on the wiki part if you'd like. Are backups made in the same format as the scripts method (eg can I basically steal that info from the scripts wiki page?) Is it possible to replicate a backup or just the live index? I also want to put in what happens when you shutdown during a replication - does Solr wait? Abort? If its aborts, are the temp files cleaned up later? replicate and polling both to default to on right? and the replicate setting just stops any slaves hitting a master with replicate=off from syncing?
        Hide
        Noble Paul added a comment -

        Lets also clearup the doc for this stuff as part of the issue....

        yes the wiki has to be cleaned up too

        Are backups made in the same format as the scripts method

        The format is same (same dir name format). The mechanism is different. scripts use hard links , this uses copying

        Is it possible to replicate a backup or just the live index?

        nope. only live index can be transferred. The backup can be made the live index and then the user can replicate it. There is a file called index.properties in the dataDir (only created by replicationhandler. but users can create them too ). It can take a property index=<dir-name> which tells solr core where to load the index from. users can edit that and do a reload core for the new backup to be used as the index.

        what happens when you shutdown during a replication - does Solr wait? Abort? If its aborts, are the temp files cleaned up later?

        replication is aborted. temp files will remain as is. They will not be cleaned up automatically. But presence of those temp files will not have any impact of future replication.

        replicate and polling both to default to on right?

        yes

        and the replicate setting just stops any slaves hitting a master with replicate=off from syncing?

        the slaves will keep polling even if the master has disabled replication . But the master will respond with "no changes" .This will ensure that when the replication is re-enabled from master , the slaves will be able to resume replication

        Show
        Noble Paul added a comment - Lets also clearup the doc for this stuff as part of the issue.... yes the wiki has to be cleaned up too Are backups made in the same format as the scripts method The format is same (same dir name format). The mechanism is different. scripts use hard links , this uses copying Is it possible to replicate a backup or just the live index? nope. only live index can be transferred. The backup can be made the live index and then the user can replicate it. There is a file called index.properties in the dataDir (only created by replicationhandler. but users can create them too ). It can take a property index=<dir-name> which tells solr core where to load the index from. users can edit that and do a reload core for the new backup to be used as the index. what happens when you shutdown during a replication - does Solr wait? Abort? If its aborts, are the temp files cleaned up later? replication is aborted. temp files will remain as is. They will not be cleaned up automatically. But presence of those temp files will not have any impact of future replication. replicate and polling both to default to on right? yes and the replicate setting just stops any slaves hitting a master with replicate=off from syncing? the slaves will keep polling even if the master has disabled replication . But the master will respond with "no changes" .This will ensure that when the replication is re-enabled from master , the slaves will be able to resume replication
        Hide
        Noble Paul added a comment -

        Changes:

        • 'snappull' renamed to 'sync'
        • 'abortsnappull' renamed to 'abortsync'
          *'snapshoot' renamed to 'backup'

        I shall commit this in a day or two

        Show
        Noble Paul added a comment - Changes: 'snappull' renamed to 'sync' 'abortsnappull' renamed to 'abortsync' *'snapshoot' renamed to 'backup' I shall commit this in a day or two
        Hide
        Walter Underwood added a comment -

        "sync" is a weak name, because it doesn't say whether it is a push or pull synchronization.

        Show
        Walter Underwood added a comment - "sync" is a weak name, because it doesn't say whether it is a push or pull synchronization.
        Hide
        Mark Miller added a comment -

        thats why I was torn between it and "syncFromMaster".

        Not in love with that either though. Any suggestions?

        Show
        Mark Miller added a comment - thats why I was torn between it and "syncFromMaster". Not in love with that either though. Any suggestions?
        Hide
        Walter Underwood added a comment -

        If we choose a name for the thing we are pulling, like "image", then we can use "makeimage", "pullimage", etc.

        Show
        Walter Underwood added a comment - If we choose a name for the thing we are pulling, like "image", then we can use "makeimage", "pullimage", etc.
        Hide
        Shalin Shekhar Mangar added a comment -

        If we choose a name for the thing we are pulling, like "image", then we can use "makeimage", "pullimage", etc.

        How about "pullIndex"?

        Show
        Shalin Shekhar Mangar added a comment - If we choose a name for the thing we are pulling, like "image", then we can use "makeimage", "pullimage", etc. How about "pullIndex"?
        Hide
        Noble Paul added a comment -

        "image' gives the same idea of snapshot. it suggests that an image of the index exists

        how about 'fetchIndex' and 'abortfetch' ?

        Show
        Noble Paul added a comment - "image' gives the same idea of snapshot. it suggests that an image of the index exists how about 'fetchIndex' and 'abortfetch' ?
        Hide
        Noble Paul added a comment -

        let us choose a name and close this soon

        Show
        Noble Paul added a comment - let us choose a name and close this soon
        Hide
        Noble Paul added a comment -

        'fetchindex 'and 'abortfetch' are the commands. I shall commit this shortly

        Show
        Noble Paul added a comment - 'fetchindex 'and 'abortfetch' are the commands. I shall commit this shortly
        Hide
        Noble Paul added a comment -

        committed : r787212

        Show
        Noble Paul added a comment - committed : r787212
        Hide
        Noble Paul added a comment -

        snapshoot command renamed to backup

        Show
        Noble Paul added a comment - snapshoot command renamed to backup
        Hide
        Noble Paul added a comment -

        commitetd r790559

        Show
        Noble Paul added a comment - commitetd r790559
        Hide
        Mark Miller added a comment -

        Comments for ReplicationHandler still need to be updated.

        Show
        Mark Miller added a comment - Comments for ReplicationHandler still need to be updated.
        Hide
        Mark Miller added a comment -

        Also, internally, everything is still SnapPuller, SpanShot for classnames and variable names and what not.

        I think we have to go whole hog if we want to change to fetchindex - otherwise, its still confusing anyway.

        We could also stick with snappull and snapshot, but I think we would need to be more clear about that means - in terms of Lucene, in terms of the old scripts, and in terms of the ReplicationHandler.

        Show
        Mark Miller added a comment - Also, internally, everything is still SnapPuller, SpanShot for classnames and variable names and what not. I think we have to go whole hog if we want to change to fetchindex - otherwise, its still confusing anyway. We could also stick with snappull and snapshot, but I think we would need to be more clear about that means - in terms of Lucene, in terms of the old scripts, and in terms of the ReplicationHandler.
        Hide
        Mark Miller added a comment -

        By the way, thanks for beefing up the wiki page Noble!

        Show
        Mark Miller added a comment - By the way, thanks for beefing up the wiki page Noble!
        Hide
        Noble Paul added a comment -

        I guess changing variable names is fine and I can do it. But if we change the classnames we lose the history altogether .

        Show
        Noble Paul added a comment - I guess changing variable names is fine and I can do it. But if we change the classnames we lose the history altogether .
        Hide
        Hoss Man added a comment -

        I guess changing variable names is fine and I can do it. But if we change the classnames we lose the history altogether .

        are you talking about svn history?

        "svn mv" preserves all history ... the only danger is in applying patches generated by someone else, you have to manually finesse applying the patch to get the right history bits flipped before committing.

        Show
        Hoss Man added a comment - I guess changing variable names is fine and I can do it. But if we change the classnames we lose the history altogether . are you talking about svn history? "svn mv" preserves all history ... the only danger is in applying patches generated by someone else, you have to manually finesse applying the patch to get the right history bits flipped before committing.
        Hide
        Noble Paul added a comment -

        "svn mv" preserves all history .

        then it is fine to rename the Classes

        • SnapPuller -> ReplicationClient
        • SnapShooter -> IndexBackup

        or if you have any other names in mind that is fine

        Show
        Noble Paul added a comment - "svn mv" preserves all history . then it is fine to rename the Classes SnapPuller -> ReplicationClient SnapShooter -> IndexBackup or if you have any other names in mind that is fine
        Hide
        Grant Ingersoll added a comment -

        Bulk close for Solr 1.4

        Show
        Grant Ingersoll added a comment - Bulk close for Solr 1.4

          People

          • Assignee:
            Noble Paul
            Reporter:
            Noble Paul
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development