Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-218

Name view files by function signature for no-downtime deployments

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10
    • Component/s: JavaScript View Server
    • Labels:
      None

      Description

      When a view is altered the .view file is deleted and then recreated. Any query (with update=false) in the interim gets no results even though the results from the old view would have been adequate.

      Consider switching the replacement view file in after it's been rebuilt like db compaction does.

        Activity

        Hide
        jchris Chris Anderson added a comment -

        Just had an IRC conversation about this today. Putting it here (mostly for my reference).

        The fix will be slightly less automatic than you mention (so we can stay transactional) but I think it will answer the use case. See transcript for details.

        basically, if we name view indexes after the view signature, then you can generate them on a non-queried ddoc, and then push those same new views to the active ddoc when the index is built, and it would just start using it. should be not hard to implement.

        the pattern (not implemented, requires a couch patch, is to deploy to a "staging" ddoc
        [2:38pm] <jchris>
        basically a ddoc that isn't used by client queries
        [2:38pm] <jchris>
        build the index
        [2:39pm] <jchris>
        and then push the new code to the production ddoc
        [2:39pm] <jchris>
        (with a small couch change I think we can make it flip to the new index transactionally)
        [2:39pm] <jchris>
        no downtime
        [2:39pm] <jchris>
        I'll write a jira ticket
        [2:40pm] <mattetti>
        that would be great, rebuilding views can be pretty slow when you have a big db
        [2:40pm] <jchris>
        targetting 0.10.0
        [2:40pm] <mattetti>
        let's get 0.9.0 out first
        [2:40pm] <jchris>
        in the mean time, version your ddoc names or something
        [2:40pm] <jchris>
        if you can't have downtime
        [2:41pm] <mattetti>
        I can have downtime right now, but I will really need conditional replication soon tho is that for 0.9 or 1.0? (or 0.10)
        [2:41pm] <jchris>
        won't make 0.9.0
        [2:41pm] <jchris>
        should be an easy change

        Show
        jchris Chris Anderson added a comment - Just had an IRC conversation about this today. Putting it here (mostly for my reference). The fix will be slightly less automatic than you mention (so we can stay transactional) but I think it will answer the use case. See transcript for details. basically, if we name view indexes after the view signature, then you can generate them on a non-queried ddoc, and then push those same new views to the active ddoc when the index is built, and it would just start using it. should be not hard to implement. the pattern (not implemented, requires a couch patch, is to deploy to a "staging" ddoc [2:38pm] <jchris> basically a ddoc that isn't used by client queries [2:38pm] <jchris> build the index [2:39pm] <jchris> and then push the new code to the production ddoc [2:39pm] <jchris> (with a small couch change I think we can make it flip to the new index transactionally) [2:39pm] <jchris> no downtime [2:39pm] <jchris> I'll write a jira ticket [2:40pm] <mattetti> that would be great, rebuilding views can be pretty slow when you have a big db [2:40pm] <jchris> targetting 0.10.0 [2:40pm] <mattetti> let's get 0.9.0 out first [2:40pm] <jchris> in the mean time, version your ddoc names or something [2:40pm] <jchris> if you can't have downtime [2:41pm] <mattetti> I can have downtime right now, but I will really need conditional replication soon tho is that for 0.9 or 1.0? (or 0.10) [2:41pm] <jchris> won't make 0.9.0 [2:41pm] <jchris> should be an easy change
        Hide
        jchris Chris Anderson added a comment -

        thinking about this patch, I realize the hard part will be deleting un-needed indexes. basically we'll need to periodically calculate the hashes of all ddocs in a database, and delete any index files who's names don't match any of the hashes. this could be triggered on ddoc update asynchronously.

        ahh, cache invalidation...

        Show
        jchris Chris Anderson added a comment - thinking about this patch, I realize the hard part will be deleting un-needed indexes. basically we'll need to periodically calculate the hashes of all ddocs in a database, and delete any index files who's names don't match any of the hashes. this could be triggered on ddoc update asynchronously. ahh, cache invalidation...
        Hide
        jchris Chris Anderson added a comment -

        as long as we're in there optimizing this stuff, we should make sure that the ddoc signature is generated the same regardless of the order of the views in the JSON document. this should be just a matter of sorting before taking the signature. it will prevent us from invalidating indexes that are actually valid.

        Show
        jchris Chris Anderson added a comment - as long as we're in there optimizing this stuff, we should make sure that the ddoc signature is generated the same regardless of the order of the views in the JSON document. this should be just a matter of sorting before taking the signature. it will prevent us from invalidating indexes that are actually valid.
        Hide
        freddybowen Freddy Bowen added a comment -

        Maybe canonical ordering of views consistent with http://github.com/jchris/canonical-json

        Show
        freddybowen Freddy Bowen added a comment - Maybe canonical ordering of views consistent with http://github.com/jchris/canonical-json
        Hide
        jchris Chris Anderson added a comment -

        Closed in r791140

        Show
        jchris Chris Anderson added a comment - Closed in r791140

          People

          • Assignee:
            jchris Chris Anderson
            Reporter:
            rnewson Robert Newson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development