Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-2804

The fast_view optimisation for view-filtered changes feeds is not cluster-aware

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: Database Core
    • Labels:
      None

      Description

      The view changes functionality (the most up-to-date documentation I am aware of being here: https://github.com/rcouch/rcouch/wiki/View-Changes) needs some work to make the fast_view optimisation work via the clustered interface.

      This has been tested using the following script: https://gist.github.com/mikewallace1979/1f53f186e48bb888c256#file-test_fast_view-sh

      Running it against the admin port on 15986 produces a successful response [1] although people with an eye for detail will notice a secondary bug where the first change isn't included.

      Running it against the clustered port on 15984 produces a stack trace [2]:

      {"error":"error","reason":"{function_clause,nil,\n    [{couch_changes,filter,\n         [{db,<26600.3277.0>,nil,<<\"1441816974537529\">>,<26600.3259.0>,\n              #Ref<26600.0.0.19015>,\n              {db_header,6,1,0,\n                  {4310,{1,0,{size_info,68,105}},96},\n                  {4406,1,93},\n                  {21426,[],1273},\n                  0,nil,nil,1000,<<\"585e0d4edddc04b8423fee965a730e4c\">>,\n                  [{'node2@127.0.0.1',0}],\n                  0},\n              1,\n              {btree,<26600.3259.0>,\n                  {4310,{1,0,{size_info,68,105}},96},\n                  #Fun<couch_db_updater.btree_by_id_split.1>,\n                  #Fun<couch_db_updater.btree_by_id_join.2>,undefined,\n                  #Fun<couch_db_updater.btree_by_id_reduce.2>,snappy},\n              {btree,<26600.3259.0>,\n                  {4406,1,93},\n                  #Fun<couch_db_updater.btree_by_seq_split.1>,\n                  #Fun<couch_db_updater.btree_by_seq_join.2>,undefined,\n                  #Fun<couch_db_updater.btree_by_seq_reduce.2>,snappy},\n              {btree,<26600.3259.0>,\n                  {21426,[],1273},\n                  undefined,undefined,undefined,nil,snappy},\n              1,<<\"shards/c0000000-dfffffff/test_db.1441816974\">>,\n              \"/Users/mikewallace/src/asf/couchdb/dev/lib/node2/data/shards/c0000000-dfffffff/test_db.1441816974.couch\",\n              undefined,[],nil,\n              {user_ctx,null,[],undefined},\n              nil,1000,\n              [before_header,after_header,on_file_open],\n              [create],\n              snappy,nil,nil},\n          {doc_info,<<\"doge\">>,1,\n              [{rev_info,\n                   {1,\n                    <<174,50,232,100,203,90,27,198,130,81,79,221,160,216,74,\n                      128>>},\n                   1,false,4242}]},\n          {fast_view,main_only,\n              {doc,<<\"_design/fastview\">>,\n                  {1,\n                   [<<8,240,229,73,29,55,237,140,209,212,55,78,252,52,73,20>>]},\n                  {[{<<\"views\">>,\n                     {[{<<\"doge\">>,\n                        {[{<<\"map\">>,\n                           <<\"function(doc) { if (true) { emit(doc.id, 1); }};\">>}]}}]}},\n                    {<<\"options\">>,{[{<<\"seq_indexed\">>,true}]}}]},\n                  [],false,[]},\n              <<\"doge\">>}],\n         [{file,\"src/couch_changes.erl\"},{line,243}]},\n     {fabric_rpc,changes_enumerator,2,\n         [{file,\"src/fabric_rpc.erl\"},{line,328}]},\n     {couch_btree,stream_kv_node2,8,[{file,\"src/couch_btree.erl\"},{line,783}]},\n     {couch_btree,fold,4,[{file,\"src/couch_btree.erl\"},{line,220}]},\n     {couch_db,changes_since,5,[{file,\"src/couch_db.erl\"},{line,1225}]},\n     {fabric_rpc,changes,4,[{file,\"src/fabric_rpc.erl\"},{line,68}]},\n     {rexi_server,init_p,3,[{file,\"src/rexi_server.erl\"},{line,139}]}]}"}
      

      So it looks like we need to do a bit of work in fabric to make it aware of the fast_view optimisation.

      [1] https://gist.github.com/mikewallace1979/1f53f186e48bb888c256#file-output_admin_port-md
      [2] https://gist.github.com/mikewallace1979/1f53f186e48bb888c256#file-output_cluster_port-md

        Activity

        Hide
        mikewallace Mike Wallace added a comment -

        In case anyone is interested, I did some (very handwavy) testing against the admin port to see how much of a speedup the fast_view optimisation gives us - I was typically getting results along these lines:

        Raw changes:               1.702s
        Filtered changes:         53.113s
        Filtered-by-view changes: 57.041s
        Fast-view changes:         9.500s
        

        So it's certainly a valuable speed up.

        (usual disclaimers apply - these figures were not obtained using science)

        Show
        mikewallace Mike Wallace added a comment - In case anyone is interested, I did some (very handwavy) testing against the admin port to see how much of a speedup the fast_view optimisation gives us - I was typically getting results along these lines: Raw changes: 1.702s Filtered changes: 53.113s Filtered-by-view changes: 57.041s Fast-view changes: 9.500s So it's certainly a valuable speed up. (usual disclaimers apply - these figures were not obtained using science)

          People

          • Assignee:
            Unassigned
            Reporter:
            mikewallace Mike Wallace
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development