[COUCHDB-1256] Incremental requests to _changes can skip revisions

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.10, 0.10.1, 0.10.2, 0.11.1, 0.11.2, 1.0, 1.0.1, 1.0.2, 1.0.3, 1.1
• Fix Version/s: 1.1.1, 1.2, 1.2.1
• Component/s: Replication

Environment

confirmed on Apache CouchDB 1.1.0, bug appears to be present in 1.0.3 and trunk

• Skill Level: Committers Level (Medium to Hard)
• Assignee: Adam Kocoloski
• Reporter: Adam Kocoloski
• Created: 22/Aug/11 14:18
• Updated: 29/May/13 15:44
• Resolved: 25/Aug/11 19:16

Description

Requests to _changes with style=all_docs&since=N (requests made by the replicator) are liable to suppress revisions of a document. The following sequence of curl commands demonstrates the bug:

curl -X PUT localhost:5985/revseq

{"ok":true}

curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/foo -d '

{"a":123}

'

{"ok":true,"id":"foo","rev":"1-0dc33db52a43872b6f3371cef7de0277"}

curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/bar -d '

{"a":456}

'

{"ok":true,"id":"bar","rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}

% stick a conflict revision in foo
curl -X PUT -Hcontent-type:application/json localhost:5985/revseq/foo?new_edits=false -d '

{"_rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a", "a":123}

'

{"ok":true,"id":"foo","rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}

% request without since= gives the expected result
curl -Hcontent-type:application/json localhost:5985/revseq/_changes?style=all_docs
{"results":[
{"seq":2,"id":"bar","changes":[

{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}

]},
{"seq":3,"id":"foo","changes":[

{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}

,

{"rev":"1-0dc33db52a43872b6f3371cef7de0277"}

]}
],
"last_seq":3}

% request starting from since=2 suppresses revision 1-0dc33db52a43872b6f3371cef7de0277 of foo
macbook:~ (master) $ curl localhost:5985/revseq/_changes?style=all_docs\&since=2
{"results":[
{"seq":3,"id":"foo","changes":[

{"rev":"1-cc609831f0ca66e8cd3d4c1e0d98108a"}

]}
],
"last_seq":3}

I believe the fix is something like this (though we could refactor further because Style is unused):

diff --git a/src/couchdb/couch_db.erl b/src/couchdb/couch_db.erl
index e8705be..65aeca3 100644
— a/src/couchdb/couch_db.erl
+++ b/src/couchdb/couch_db.erl
@@ -1029,19 +1029,7 @@ changes_since(Db, Style, StartSeq, Fun, Acc) ->
changes_since(Db, Style, StartSeq, Fun, [], Acc).

changes_since(Db, Style, StartSeq, Fun, Options, Acc) ->

• Wrapper = fun(DocInfo, _Offset, Acc2) ->
• #doc_info {revs=Revs}

  = DocInfo,

• DocInfo2 =
• case Style of
• main_only ->
• DocInfo;
• all_docs ->
• % remove revs before the seq
• DocInfo#doc_info{revs=[RevInfo ||
• #rev_info {seq=RevSeq}

  =RevInfo <- Revs, StartSeq < RevSeq]}

• end,
• Fun(DocInfo2, Acc2)
• end,
  + Wrapper = fun(DocInfo, _Offset, Acc2) -> Fun(DocInfo, Acc2) end,
  {ok, _LastReduction, AccOut}

  = couch_btree:fold(by_seq_btree(Db),
  Wrapper, Acc, [

  {start_key, StartSeq + 1}

  ] ++ Options),
  

  {ok, AccOut}

  .