CouchDB
  1. CouchDB
  2. COUCHDB-494

Add include_seqs=true option to _all_dbs to allow efficient bootstrapping of external indexes.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Database Core
    • Labels:
      None
    • Skill Level:
      New Contributors Level (Easy)

      Description

      External indexers need an efficient way to detect changes across all databases when they boot up. The _all_dbs endpoint only lists the names. We propose adding include_seqs=true which would return this style output instead;

      [

      {"name":"db1","update_seq",5000}

      ,

      {"name":"db2","update_seq",2500}

      ]

      this would allow an external indexer to skip any database that has not changed.

      As external indexers often store their configuration changes inside design documents, it might also be helpful if the update_seq of the latest change to any design document was also included. This will allow external indexers to skip even databases with changes as long as those changes are not to design documents;

      [

      {"name":"db1","update_seq",5000, "ddoc_update_seq":23}

      ,

      {"name":"db2","update_seq",2500, "ddoc_update_seq":2500}

      ]

        Issue Links

          Activity

          Hide
          Robert Newson added a comment -


          Also consider feed=continuous and feed=longpoll support

          Show
          Robert Newson added a comment - Also consider feed=continuous and feed=longpoll support
          Hide
          Antony Blakey added a comment -

          This should include the db's UUID, which has recently been added as a more reliable way to identify a database.

          Show
          Antony Blakey added a comment - This should include the db's UUID, which has recently been added as a more reliable way to identify a database.
          Hide
          Dipesh Patel added a comment -

          Hi!

          Is this still an issue? I'd like to work on it if so but it looks like all the info can be gained from the _changes feed?

          Dip

          Show
          Dipesh Patel added a comment - Hi! Is this still an issue? I'd like to work on it if so but it looks like all the info can be gained from the _changes feed? Dip
          Hide
          Adam Kocoloski added a comment -

          Hi, it hasn't been resolved. It's a bit problematic, though. The current implementation of _all_dbs has the nice property that it doesn't actually open any of the databases when it generates the response. If we added the sequence numbers I think we'd also need to add pagination to avoid an attempt to open 1000s of databases in rapid succession. I also think this needs to be opt-in to avoid making a breaking change to the API.

          Show
          Adam Kocoloski added a comment - Hi, it hasn't been resolved. It's a bit problematic, though. The current implementation of _all_dbs has the nice property that it doesn't actually open any of the databases when it generates the response. If we added the sequence numbers I think we'd also need to add pagination to avoid an attempt to open 1000s of databases in rapid succession. I also think this needs to be opt-in to avoid making a breaking change to the API.
          Hide
          Dipesh Patel added a comment -

          Hi Adam

          Yeah I noticed this when I looked into it. In order to get this information we will have to open the dbs which of course has a performance impact like you mention. I'll take a stab and implement with some form of pagination, I think I have something working with an opt-in.

          Dip

          Show
          Dipesh Patel added a comment - Hi Adam Yeah I noticed this when I looked into it. In order to get this information we will have to open the dbs which of course has a performance impact like you mention. I'll take a stab and implement with some form of pagination, I think I have something working with an opt-in. Dip
          Hide
          Jan Lehnardt added a comment -

          With COUCHDB-360 in place (in the future), this won't be needed anymore. In addition, the proposed solution is sub-optimal and COUCHDB-360 solves the problem at hand more elegantly.

          Show
          Jan Lehnardt added a comment - With COUCHDB-360 in place (in the future), this won't be needed anymore. In addition, the proposed solution is sub-optimal and COUCHDB-360 solves the problem at hand more elegantly.

            People

            • Assignee:
              Unassigned
              Reporter:
              Robert Newson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development