CouchDB
  1. CouchDB
  2. COUCHDB-523

View API POST keys to retrieve multiple docs by key could also allow for multiple 'range' queries, i.e. an array of { startkey: .., endkey: ... } params in the POST

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: HTTP Interface
    • Labels:
      None
    • Skill Level:
      Committers Level (Medium to Hard)

      Description

      It would be useful if I could do a single POST to a view to retrieve multiple ranges specified by startkey, endkey.

      The format could be as follows:

      { "ranges": [

      { "startkey": "a", "endkey": "c" }

      ,

      { "startkey":"g", "endkey":"z" }

      ] }

      1. couch_httpd_view.erl
        27 kB
        Jamie Talbot
      2. multi_start_end_key.diff
        2 kB
        Jamie Talbot
      3. ranged_key_post.diff
        0.8 kB
        Benjamin Anderson

        Issue Links

          Activity

          Hide
          Jamie Talbot added a comment -

          I wanted this feature so much, I hacked up a version that appears to do it. I'm nowhere near an Erlang or Couch expert, but it seems to do what I want and still be lightning fast so perhaps it is a solution. The only changes were made to couch_httpd_view.erl. General theory of operation is to pattern match start and end key fields from the Key variable in output_map_view and output_reduce_view. I also had to list the restrictions on group_levels for multi-key documents seeing as they do make sense in this context.

          Was pretty much scrambling in the dark, but If that sounds like a sensible approach, and someone can give me pointers on how to contribute, I'd be happy to attach it here.

          In the meantime, I'll attach my hacked up version of couch_http_view.erl. I've also included a diff from version 0.10.1.

          Show
          Jamie Talbot added a comment - I wanted this feature so much, I hacked up a version that appears to do it. I'm nowhere near an Erlang or Couch expert, but it seems to do what I want and still be lightning fast so perhaps it is a solution. The only changes were made to couch_httpd_view.erl. General theory of operation is to pattern match start and end key fields from the Key variable in output_map_view and output_reduce_view. I also had to list the restrictions on group_levels for multi-key documents seeing as they do make sense in this context. Was pretty much scrambling in the dark, but If that sounds like a sensible approach, and someone can give me pointers on how to contribute, I'd be happy to attach it here. In the meantime, I'll attach my hacked up version of couch_http_view.erl. I've also included a diff from version 0.10.1.
          Hide
          Jamie Talbot added a comment -

          Hacked version of couchdb_httpd_view.erl as of 0.10.1, to include multi start and end key functionality.

          Also a diff from 0.10.1.

          Show
          Jamie Talbot added a comment - Hacked version of couchdb_httpd_view.erl as of 0.10.1, to include multi start and end key functionality. Also a diff from 0.10.1.
          Hide
          Nikita Nemkin added a comment -

          This is a very useful feature when indexing spatial data using z-order curve, hilbert curve etc. Queries to such indexes translate directly to a bunch of B-tree range queries (up to hundreds of ranges, depending on implementation).

          Show
          Nikita Nemkin added a comment - This is a very useful feature when indexing spatial data using z-order curve, hilbert curve etc. Queries to such indexes translate directly to a bunch of B-tree range queries (up to hundreds of ranges, depending on implementation).
          Hide
          Jamie Talbot added a comment -

          I should mention that the supplied patch allows the setting of a group_level, but doesn't aggregate across the differing ranges. I.e. Say I had a blog and my "post count" view had a map to output "1" for every post and a reduce to sum those values, querying it with:

          {
          "keys": [
          {
          "startkey": ["Category A","2010","03"],
          "endkey": ["Category A","2010","04",{}]
          },
          {
          "startkey": ["Category A","2010","06"],
          "endkey": ["Category A","2010","07",{}]
          }
          ]
          }

          using group=true&group_level=1

          still gives me 2 rows:
          Category A : 12
          Category A : 14

          (for example). Would be nice if it gave me one row of 26.

          I think it's because the ranges are processed separately (more of a convenience than anything). Someone more knowledgeable than me can correct me if I'm wrong.

          Show
          Jamie Talbot added a comment - I should mention that the supplied patch allows the setting of a group_level, but doesn't aggregate across the differing ranges. I.e. Say I had a blog and my "post count" view had a map to output "1" for every post and a reduce to sum those values, querying it with: { "keys": [ { "startkey": ["Category A","2010","03"] , "endkey": ["Category A","2010","04",{}] }, { "startkey": ["Category A","2010","06"] , "endkey": ["Category A","2010","07",{}] } ] } using group=true&group_level=1 still gives me 2 rows: Category A : 12 Category A : 14 (for example). Would be nice if it gave me one row of 26. I think it's because the ranges are processed separately (more of a convenience than anything). Someone more knowledgeable than me can correct me if I'm wrong.
          Hide
          Walt W added a comment -

          I don't actually have the time to test this at the moment, but I wanted to give this issue my vote and mention that this is a VERY useful feature . . . I'm not sure it should be included in a release without fixing the issue Jamie mentioned (grouping across different ranges), but if that could be fixed . . . this would be incredibly useful, and cut down on the space needed to make a bunch of different views to emulate this.

          Show
          Walt W added a comment - I don't actually have the time to test this at the moment, but I wanted to give this issue my vote and mention that this is a VERY useful feature . . . I'm not sure it should be included in a release without fixing the issue Jamie mentioned (grouping across different ranges), but if that could be fixed . . . this would be incredibly useful, and cut down on the space needed to make a bunch of different views to emulate this.
          Hide
          Benjamin Anderson added a comment -

          This would be a big win for me as well, so I made a quick hack of the current trunk (1.1.0a961514) to enable it. It's pretty close to Jamie's work, except I didn't mess with the group_level code. Diff is attached.

          My tests indicate that it works great, but they're admittedly limited; if there are any test suites out there that I can throw at it, let me know.

          Similar to Jamie, I'm a novice with Erlang, but I'm a heavy Couch user and would like to get involved. If there's any change I can make to improve this, I'd be glad to spend the time on it.

          Show
          Benjamin Anderson added a comment - This would be a big win for me as well, so I made a quick hack of the current trunk (1.1.0a961514) to enable it. It's pretty close to Jamie's work, except I didn't mess with the group_level code. Diff is attached. My tests indicate that it works great, but they're admittedly limited; if there are any test suites out there that I can throw at it, let me know. Similar to Jamie, I'm a novice with Erlang, but I'm a heavy Couch user and would like to get involved. If there's any change I can make to improve this, I'd be glad to spend the time on it.
          Hide
          Adam Kocoloski added a comment -

          Seems pretty useful to me. I think re-reducing the final output is not too hard if the key ranges do not overlap. But what happens when they do? I suppose a clever thing to do would be to merge the overlapping ranges, but that doesn't strike me as particularly relaxed.

          Show
          Adam Kocoloski added a comment - Seems pretty useful to me. I think re-reducing the final output is not too hard if the key ranges do not overlap. But what happens when they do? I suppose a clever thing to do would be to merge the overlapping ranges, but that doesn't strike me as particularly relaxed.
          Hide
          Adam Kocoloski added a comment -

          I've been thinking about this issue a bit and would like to propose an alternative syntax. Instead of relying on overloading the "keys" field with things that are not really view keys, we could allow a new field called "queries" that looks like

          {"queries":[

          {"key":"foo"}

          ,

          {"startkey":"bar", "endkey":"baz", "limit":10}

          , ...]}

          That is, each element of the queries Array would be a JSON Object specifying view query parameters. Any parameters specified in the actual query string would be included as defaults. CouchDB would execute the view requests serially and respond with an Array of view responses

          [

          {"total_rows":100, "offset":34, "rows":[...]}

          ,

          {"total_rows":100, "offset":20, "rows":[...]}

          ]

          The "keys" field and the "queries" field would be mutually exclusive. The Objects in the queries Array would allow a user to specify any query-string parameter used with CouchDB views. I'm not sure if _list functions would be supported.

          This feature is really a crutch to help HTTP clients that cannot avail themselves of advanced HTTP features such as pipelining. I'm happy to add it, though, as that the set of clients which do support pipelining is not that large.

          What do you think?

          Show
          Adam Kocoloski added a comment - I've been thinking about this issue a bit and would like to propose an alternative syntax. Instead of relying on overloading the "keys" field with things that are not really view keys, we could allow a new field called "queries" that looks like {"queries":[ {"key":"foo"} , {"startkey":"bar", "endkey":"baz", "limit":10} , ...]} That is, each element of the queries Array would be a JSON Object specifying view query parameters. Any parameters specified in the actual query string would be included as defaults. CouchDB would execute the view requests serially and respond with an Array of view responses [ {"total_rows":100, "offset":34, "rows":[...]} , {"total_rows":100, "offset":20, "rows":[...]} ] The "keys" field and the "queries" field would be mutually exclusive. The Objects in the queries Array would allow a user to specify any query-string parameter used with CouchDB views. I'm not sure if _list functions would be supported. This feature is really a crutch to help HTTP clients that cannot avail themselves of advanced HTTP features such as pipelining. I'm happy to add it, though, as that the set of clients which do support pipelining is not that large. What do you think?
          Hide
          Anand Chitipothu added a comment -

          How about considering a dict instead of a list for queries? Keeping track of dictionary keys is much easier than keeping track of array indicies.

          {
          "queries": {
          "foo":

          {"key":"foo"}

          ,
          "bar":

          {"startkey":"bar", "endkey":"baz", "limit":10}

          ,
          ...
          }
          }

          And the response will be dictionary with same keys.

          {
          "results": {
          "foo":

          {"total_rows":100, "offset":34, "rows":[...]}

          ,
          "bar":

          {"total_rows":100, "offset":20, "rows":[...]}


          }
          }

          I'm not very sure if introducing another nested level "results" is required, but it looks symmetrical to the request.

          Show
          Anand Chitipothu added a comment - How about considering a dict instead of a list for queries? Keeping track of dictionary keys is much easier than keeping track of array indicies. { "queries": { "foo": {"key":"foo"} , "bar": {"startkey":"bar", "endkey":"baz", "limit":10} , ... } } And the response will be dictionary with same keys. { "results": { "foo": {"total_rows":100, "offset":34, "rows":[...]} , "bar": {"total_rows":100, "offset":20, "rows":[...]} } } I'm not very sure if introducing another nested level "results" is required, but it looks symmetrical to the request.
          Hide
          Adam Kocoloski added a comment -

          Hi Anand, I'd like to see some of the other commenters weigh in on that one. Personally I think it'd be a bit of a hassle to name each of my queries in a multi-query request. I think I prefer the order-preserving behavior of an Array, where I could construct my queries Array in order and then just concatenate the rows from the individual responses.

          I can say that implementing the API as you've written it is not much more difficult than the Array-based one.

          Show
          Adam Kocoloski added a comment - Hi Anand, I'd like to see some of the other commenters weigh in on that one. Personally I think it'd be a bit of a hassle to name each of my queries in a multi-query request. I think I prefer the order-preserving behavior of an Array, where I could construct my queries Array in order and then just concatenate the rows from the individual responses. I can say that implementing the API as you've written it is not much more difficult than the Array-based one.
          Hide
          Paul Joseph Davis added a comment -

          I would say that Adam's array based approach is probably going to be more sensible as it allows people to more easily control the order in which the queries are processed in the case where someone wants to aggregate some sort of result across the multiple views.

          For the response I think having a

          {"results": $array_of_results}

          wrapper is probably better as it allows for future changes more easily if we start annotating that response.

          Just for fun, has anyone considered how this would integrate with _lists? Or if we people should be allowed to make queries on multiple views, and if so if they should or shouldn't be scoped to a design doc?

          Show
          Paul Joseph Davis added a comment - I would say that Adam's array based approach is probably going to be more sensible as it allows people to more easily control the order in which the queries are processed in the case where someone wants to aggregate some sort of result across the multiple views. For the response I think having a {"results": $array_of_results} wrapper is probably better as it allows for future changes more easily if we start annotating that response. Just for fun, has anyone considered how this would integrate with _lists? Or if we people should be allowed to make queries on multiple views, and if so if they should or shouldn't be scoped to a design doc?
          Hide
          Jan Lehnardt added a comment -

          Still in heavy discussion, bumping the fix version from 1.1 to unknown.

          Show
          Jan Lehnardt added a comment - Still in heavy discussion, bumping the fix version from 1.1 to unknown.
          Hide
          Christian Carter added a comment -

          This feature would give a lot of amazing flexibility for pure couchapps, especially if you could decide reduce behavior (separate reduces vs. one reduce) and you could use a list function on it. But even if those aren't possible, seeing it in the next release would be fantastic!

          Show
          Christian Carter added a comment - This feature would give a lot of amazing flexibility for pure couchapps, especially if you could decide reduce behavior (separate reduces vs. one reduce) and you could use a list function on it. But even if those aren't possible, seeing it in the next release would be fantastic!
          Hide
          Robert Newson added a comment -

          Let's reopen this and try to get it done for 1.3. I don't see any fundamental objection just bikeshedding over the API.

          Show
          Robert Newson added a comment - Let's reopen this and try to get it done for 1.3. I don't see any fundamental objection just bikeshedding over the API.
          Hide
          Robert Newson added a comment -

          also +1 on the array/order preserving API and -1 on the object thing, just to get things moving.

          Show
          Robert Newson added a comment - also +1 on the array/order preserving API and -1 on the object thing, just to get things moving.
          Hide
          Ludovic Patey added a comment -

          Could there be a preview of the future syntax ? I'll probably apply a patch on my database and would it like to be forward-compatible with the next version.

          Show
          Ludovic Patey added a comment - Could there be a preview of the future syntax ? I'll probably apply a patch on my database and would it like to be forward-compatible with the next version.
          Hide
          Karel added a comment -

          This seems like possible workaround for the absence of wildcards in compound keys of a view. E.g. now you cannot query a view like this: key=[dimension1,*,dimension3]. Obviously this is just how btree works.

          Show
          Karel added a comment - This seems like possible workaround for the absence of wildcards in compound keys of a view. E.g. now you cannot query a view like this: key= [dimension1,*,dimension3] . Obviously this is just how btree works.
          Hide
          Adam Kocoloski added a comment -

          Will rebase my old patch on master and post it for review now.

          Show
          Adam Kocoloski added a comment - Will rebase my old patch on master and post it for review now.
          Hide
          ASF IRC Bot added a comment -

          Comment from kocolosk via IRC:
          the rebase is a bit involved since the multi-query code I have predates the couch_index / couch_mrview refactor ...

          Show
          ASF IRC Bot added a comment - Comment from kocolosk via IRC: the rebase is a bit involved since the multi-query code I have predates the couch_index / couch_mrview refactor ...
          Hide
          Joan Touzet added a comment -

          Stealing back with Adam's permission.

          Show
          Joan Touzet added a comment - Stealing back with Adam's permission.
          Hide
          Amza OUSSEY added a comment -

          Hello,
          any patch for current version?
          I started learning erlang yesterday for that purpose.

          Show
          Amza OUSSEY added a comment - Hello, any patch for current version? I started learning erlang yesterday for that purpose.
          Hide
          Joan Touzet added a comment -

          Hi Amza, thanks for your offer of help.

          This is a fairly difficult bug, not one I'd recommend for a new Erlanger. I'm currently working on COUCHDB-1334 and will be moving to this bug immediately after that.

          Please be patient

          Show
          Joan Touzet added a comment - Hi Amza, thanks for your offer of help. This is a fairly difficult bug, not one I'd recommend for a new Erlanger. I'm currently working on COUCHDB-1334 and will be moving to this bug immediately after that. Please be patient
          Hide
          ASF subversion and git services added a comment -

          Commit 22ec9d906251010eced428765d24ba883e511b91 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=22ec9d9 ]

          WIP: support multi query views

          COUCHDB-523
          COUCHDB-1993

          This is a functional implementation of multi query views on the single
          node interface. This is a WIP commit as the implementation is a bit
          awkward in some places. In particular, this uses the
          couch_mrview_http:view_cb/2 for the entirety of the http response,
          which is nice, but it requires introducing state to indicate when
          we've reached the last view query, to know when we can complete the
          chunked response. Is this better than having multi_query_view/5 finish
          the final http chunk and end the response? Pros and cons to either
          approach. Once I get some feedback on approach there I'll update
          fabric to support this as well.

          Show
          ASF subversion and git services added a comment - Commit 22ec9d906251010eced428765d24ba883e511b91 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=22ec9d9 ] WIP: support multi query views COUCHDB-523 COUCHDB-1993 This is a functional implementation of multi query views on the single node interface. This is a WIP commit as the implementation is a bit awkward in some places. In particular, this uses the couch_mrview_http:view_cb/2 for the entirety of the http response, which is nice, but it requires introducing state to indicate when we've reached the last view query, to know when we can complete the chunked response. Is this better than having multi_query_view/5 finish the final http chunk and end the response? Pros and cons to either approach. Once I get some feedback on approach there I'll update fabric to support this as well.
          Hide
          ASF subversion and git services added a comment -

          Commit 017430dc393a7b36a83e0e8faa63b80859e4bc2f in couchdb-fabric's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=017430d ]

          HACK: send a meta callback from fabric_view_reduce:handle_message/3

          Hack alert. This sends a meta row from the handle_message callback in
          fabric_view_reduce for the case of receiving a total_and_offset
          messsage. Prior to couch_mrview, the view reduce implementation did
          not send this message to fabric_view_reduce.

          The hack is that we don't do the proper row collection worker dance
          like we do in the handle_message callback for #view_row. I added a
          temporary field to the #collector record to keep track of whether
          we've sent the meta row yet to ensure it only gets sent once.

          The primary motivation for this hack is that it provides a clean and
          elegant solution to understanding when a view output stream is
          starting. This is used to cleanup unecessary code in
          couch_mrview_http:view_cb/2, and it's also essential for doing multi
          query views through fabric.

          I strongly feel sending the meta row is the right general approach,
          but the devil is in the details as usual. To do this properly, we need
          to funnel this through fabric_view:maybe_send_row, which gets awkward
          in a hurry. So I've decided to temporarily introduce this hack to
          solidify the API allowing the multi_query_view implementation to be
          completed. Next step is to figure out how to fix this hack.

          COUCHDB-523
          COUCHDB-1993

          Show
          ASF subversion and git services added a comment - Commit 017430dc393a7b36a83e0e8faa63b80859e4bc2f in couchdb-fabric's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=017430d ] HACK: send a meta callback from fabric_view_reduce:handle_message/3 Hack alert. This sends a meta row from the handle_message callback in fabric_view_reduce for the case of receiving a total_and_offset messsage. Prior to couch_mrview, the view reduce implementation did not send this message to fabric_view_reduce. The hack is that we don't do the proper row collection worker dance like we do in the handle_message callback for #view_row. I added a temporary field to the #collector record to keep track of whether we've sent the meta row yet to ensure it only gets sent once. The primary motivation for this hack is that it provides a clean and elegant solution to understanding when a view output stream is starting. This is used to cleanup unecessary code in couch_mrview_http:view_cb/2, and it's also essential for doing multi query views through fabric. I strongly feel sending the meta row is the right general approach, but the devil is in the details as usual. To do this properly, we need to funnel this through fabric_view:maybe_send_row, which gets awkward in a hurry. So I've decided to temporarily introduce this hack to solidify the API allowing the multi_query_view implementation to be completed. Next step is to figure out how to fix this hack. COUCHDB-523 COUCHDB-1993
          Hide
          ASF subversion and git services added a comment -

          Commit b78d2e78cae88116bf89a3a16735e86ab5bea228 in couchdb-chttpd's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=b78d2e7 ]

          Update chttpd_view to use couch_mrview for multi query views

          This switches to using the couch_mrview implementation for handling
          multi query view requests, both for fetching the views through fabric
          and also the http callbacks.

          One concern I have with this implementation that needs to be tested,
          is whether calling couch_mrview_util:get_view in multi_query_view is
          appropriate. What happens when the database or the ddoc does not exist
          on the node handling this request? I think the ddoc should be fine as
          we load that from the ddoc_cache, but I'm less sure about the db. We
          need to make this request so we can perform the validations on the
          view, both for the url query params, and also for every set of
          additional params provided in the list of queries.

          COUCHDB-523
          COUCHDB-1993

          Show
          ASF subversion and git services added a comment - Commit b78d2e78cae88116bf89a3a16735e86ab5bea228 in couchdb-chttpd's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=b78d2e7 ] Update chttpd_view to use couch_mrview for multi query views This switches to using the couch_mrview implementation for handling multi query view requests, both for fetching the views through fabric and also the http callbacks. One concern I have with this implementation that needs to be tested, is whether calling couch_mrview_util:get_view in multi_query_view is appropriate. What happens when the database or the ddoc does not exist on the node handling this request? I think the ddoc should be fine as we load that from the ddoc_cache, but I'm less sure about the db. We need to make this request so we can perform the validations on the view, both for the url query params, and also for every set of additional params provided in the list of queries. COUCHDB-523 COUCHDB-1993
          Hide
          ASF subversion and git services added a comment -

          Commit 3015c8554ae38fc3fa7890209c5e394e5995e1c8 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=3015c85 ]

          Add support for multi query views

          This adds support for making multiple view queries in one request by
          supplying a "queries" list in a POST request body. The primary
          implementation was ported over from chttpd_view:multi_query_view/5.

          The support added here is two fold. First, we add multi_query_view
          support to the single node interface in couch_mrview_http. Second we
          add support in couch_mrview_http:view_cb/2 to allow chttpd to utilize
          this callback as well.

          This also switches updates the #vacc record from setting a multi_query
          flag, to instead setting a should_close flag. Now
          couch_mrview_http:view_cb/2 will only start a new delayed response if
          #vacc.resp is undefined, and similarly, it will only close a delayed
          response if #vacc.should_close is true.

          COUCHDB-523
          COUCHDB-1993

          Show
          ASF subversion and git services added a comment - Commit 3015c8554ae38fc3fa7890209c5e394e5995e1c8 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=3015c85 ] Add support for multi query views This adds support for making multiple view queries in one request by supplying a "queries" list in a POST request body. The primary implementation was ported over from chttpd_view:multi_query_view/5. The support added here is two fold. First, we add multi_query_view support to the single node interface in couch_mrview_http. Second we add support in couch_mrview_http:view_cb/2 to allow chttpd to utilize this callback as well. This also switches updates the #vacc record from setting a multi_query flag, to instead setting a should_close flag. Now couch_mrview_http:view_cb/2 will only start a new delayed response if #vacc.resp is undefined, and similarly, it will only close a delayed response if #vacc.should_close is true. COUCHDB-523 COUCHDB-1993
          Hide
          ASF subversion and git services added a comment -

          Commit aeac2dfe5014dc9fe1f10300669ea9244cae3a67 in couchdb-fabric's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=aeac2df ]

          Call fabric view reduce callback with meta information

          This is a temporary hack to allow passing meta through
          fabric_view_reduce:handle_message. The meta rows were not present on
          reduce functions in the previous views implementation, but they
          provide an excellent place to determine which a view is about to start
          streaming, which is essential for the multi query view implementation.

          The hack is that we don't do the proper row collection worker dance
          like we do in the handle_message callback for #view_row. For now this
          sets a process dict flag to indicate meta has been sent so that we
          only send it once. The proper fix is to send meta through
          fabric_view:maybe_send_row, but that gets ugly in a hurry and can be
          addressed separately.

          COUCHDB-523
          COUCHDB-1993

          Show
          ASF subversion and git services added a comment - Commit aeac2dfe5014dc9fe1f10300669ea9244cae3a67 in couchdb-fabric's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-fabric.git;h=aeac2df ] Call fabric view reduce callback with meta information This is a temporary hack to allow passing meta through fabric_view_reduce:handle_message. The meta rows were not present on reduce functions in the previous views implementation, but they provide an excellent place to determine which a view is about to start streaming, which is essential for the multi query view implementation. The hack is that we don't do the proper row collection worker dance like we do in the handle_message callback for #view_row. For now this sets a process dict flag to indicate meta has been sent so that we only send it once. The proper fix is to send meta through fabric_view:maybe_send_row, but that gets ugly in a hurry and can be addressed separately. COUCHDB-523 COUCHDB-1993
          Hide
          ASF subversion and git services added a comment -

          Commit 23c16c0ffbcb8a79fb4cf3969b3b698929642597 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=23c16c0 ]

          Allow couch_mrview_http to be more usable in chttpd

          This makes a number of updates to useful utility functions in
          couch_mrview, especially around parsing of requests, handling view
          rows, and interacting with view source code. The `parse_qs` function
          was updated to allow for parsing params from json bodies in addition
          to just query strings.

          The view_cb function is also updated to allow for reuse in chttpd, and
          also for better flow control of when to start and complete chunked
          responses, which is critical for multi view queries in COUCHDB-523.

          Show
          ASF subversion and git services added a comment - Commit 23c16c0ffbcb8a79fb4cf3969b3b698929642597 in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=23c16c0 ] Allow couch_mrview_http to be more usable in chttpd This makes a number of updates to useful utility functions in couch_mrview, especially around parsing of requests, handling view rows, and interacting with view source code. The `parse_qs` function was updated to allow for parsing params from json bodies in addition to just query strings. The view_cb function is also updated to allow for reuse in chttpd, and also for better flow control of when to start and complete chunked responses, which is critical for multi view queries in COUCHDB-523 .
          Hide
          ASF subversion and git services added a comment -

          Commit 3688736b81c8b8b6485cb136eea836bd729d152f in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=3688736 ]

          Add mutli view query support

          COUCHDB-523

          Show
          ASF subversion and git services added a comment - Commit 3688736b81c8b8b6485cb136eea836bd729d152f in couchdb-couch-mrview's branch refs/heads/1993-bigcouch-couch-mrview from Russell Branca [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch-mrview.git;h=3688736 ] Add mutli view query support COUCHDB-523

            People

            • Assignee:
              Russell Branca
              Reporter:
              Nathan Stott
            • Votes:
              27 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

              • Created:
                Updated:

                Development