CouchDB
  1. CouchDB
  2. COUCHDB-1365

Fix merging of document with attachment stubs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2, 1.2.1
    • Component/s: Database Core, Replication
    • Labels:
      None

      Description

      This issue was found by Marty Schoch and is reproducible the following attached script.

      The commit message in the patch explains the issue:

      "During replicated updates, merging of documents with
      attachment stubs will fail if, after merging the received
      document's revision tree with the current on disk revision
      tree, produces a revision tree which doesn't contain the revision
      that immediately precedes the received document's revision.

      This happens when the received document doesn't contain in its
      revision history any of the revisions in the revision tree
      of the currently on disk document. This is possible when the
      document had a number of updates higher than the value of revs
      limit defined for the source database."

        Activity

        Hide
        Paul Joseph Davis added a comment -

        I don't get it. Does this patch fix replication or make it fail with an error?

        Show
        Paul Joseph Davis added a comment - I don't get it. Does this patch fix replication or make it fail with an error?
        Hide
        Filipe Manana added a comment -

        Paul, it makes the merging throw a missing_stubs error instead of crashing with a function_clause like this:

        {"error":"{worker_died,<0.4681.0>,\n {function_clause,\n [{couch_doc,merge_stubs,\n [{doc,<<\"document\">>,\n

        {12,\n [<<173,196,77,158,203,221,181,67,212,246,74,43,219,237,94,\n 140>>,\n <<72,74,171,180,242,101,33,172,166,195,136,125,231,134,65,\n 37>>,\n <<83,182,225,95,63,211,178,52,96,92,102,114,130,224,138,145>>,\n <<95,234,29,149,2,72,24,226,188,255,99,148,120,126,85,103>>,\n <<87,169,231,120,244,95,82,97,203,2,50,37,173,50,61,2>>,\n <<215,75,114,100,77,137,213,77,75,174,41,30,112,205,156,95>>,\n <<65,124,138,146,230,4,155,92,52,14,189,152,167,30,231,96>>,\n <<68,16,172,15,172,224,254,189,163,203,141,251,172,187,194,\n 27>>,\n <<34,8,58,215,190,139,123,102,40,55,20,84,179,249,85,8>>,\n <<29,39,81,231,191,249,211,32,50,145,38,97,78,100,102,188>>,\n <<88,5,136,102,228,34,222,205,62,252,107,44,97,23,101,81>>,\n <<222,52,110,92,212,100,161,137,190,163,109,245,255,233,\n 85,71>>]}

        ,\n <<131,104,1,108,0,0,0,1,104,2,109,0,0,0,4,110,97,109,101,109,\n 0,0,0,6,99,98,109,97,50,50,106>>,\n [

        {att,<<\"./reproduce-CBMA-22.sh\">>,<<\"text/plain\">>,41,21,\n <<47,253,88,99,174,109,103,33,178,178,139,137,178,159,\n 243,29>>,\n 2,stub,gzip}

        ],\n false,[]},\n nil]},\n

        {couch_db,'-prep_and_validate_replicated_updates/5-fun-5-',4}

        ,\n

        {lists,foldl,3}

        ,\n

        {couch_db,prep_and_validate_replicated_updates,5}

        ,\n

        {couch_db,update_docs,4}

        ,\n

        {couch_db,update_doc,4}

        ,\n

        {couch_replicator_worker,flush_doc,2}

        ,\n

        {couch_replicator_worker,local_doc_handler,2}

        ]}}"}

        When the replicator catches a missing_stubs error, it retries replicating the document but without incremental attachment replication (sends all attachments, just like in 1.1 and below).

        Show
        Filipe Manana added a comment - Paul, it makes the merging throw a missing_stubs error instead of crashing with a function_clause like this: {"error":"{worker_died,<0.4681.0>,\n {function_clause,\n [{couch_doc,merge_stubs,\n [{doc,<<\"document\">>,\n {12,\n [<<173,196,77,158,203,221,181,67,212,246,74,43,219,237,94,\n 140>>,\n <<72,74,171,180,242,101,33,172,166,195,136,125,231,134,65,\n 37>>,\n <<83,182,225,95,63,211,178,52,96,92,102,114,130,224,138,145>>,\n <<95,234,29,149,2,72,24,226,188,255,99,148,120,126,85,103>>,\n <<87,169,231,120,244,95,82,97,203,2,50,37,173,50,61,2>>,\n <<215,75,114,100,77,137,213,77,75,174,41,30,112,205,156,95>>,\n <<65,124,138,146,230,4,155,92,52,14,189,152,167,30,231,96>>,\n <<68,16,172,15,172,224,254,189,163,203,141,251,172,187,194,\n 27>>,\n <<34,8,58,215,190,139,123,102,40,55,20,84,179,249,85,8>>,\n <<29,39,81,231,191,249,211,32,50,145,38,97,78,100,102,188>>,\n <<88,5,136,102,228,34,222,205,62,252,107,44,97,23,101,81>>,\n <<222,52,110,92,212,100,161,137,190,163,109,245,255,233,\n 85,71>>]} ,\n <<131,104,1,108,0,0,0,1,104,2,109,0,0,0,4,110,97,109,101,109,\n 0,0,0,6,99,98,109,97,50,50,106>>,\n [ {att,<<\"./reproduce-CBMA-22.sh\">>,<<\"text/plain\">>,41,21,\n <<47,253,88,99,174,109,103,33,178,178,139,137,178,159,\n 243,29>>,\n 2,stub,gzip} ],\n false,[]},\n nil]},\n {couch_db,'-prep_and_validate_replicated_updates/5-fun-5-',4} ,\n {lists,foldl,3} ,\n {couch_db,prep_and_validate_replicated_updates,5} ,\n {couch_db,update_docs,4} ,\n {couch_db,update_doc,4} ,\n {couch_replicator_worker,flush_doc,2} ,\n {couch_replicator_worker,local_doc_handler,2} ]}}"} When the replicator catches a missing_stubs error, it retries replicating the document but without incremental attachment replication (sends all attachments, just like in 1.1 and below).
        Hide
        Paul Joseph Davis added a comment -

        Ah. Sounds good to me then.

        Show
        Paul Joseph Davis added a comment - Ah. Sounds good to me then.
        Hide
        Filipe Manana added a comment -

        Actually the commit message has a small mistake:

        "This is possible when the
        document had a number of updates higher than the value of revs
        limit defined for the source database."

        Here I didn't want to say source but target database instead.

        Show
        Filipe Manana added a comment - Actually the commit message has a small mistake: "This is possible when the document had a number of updates higher than the value of revs limit defined for the source database." Here I didn't want to say source but target database instead.
        Hide
        Randall Leeds added a comment -

        +1. Mind if I lower the message about refetching in couch_replicator_worker (due to missing_stub) to WARN from ERROR?

        Show
        Randall Leeds added a comment - +1. Mind if I lower the message about refetching in couch_replicator_worker (due to missing_stub) to WARN from ERROR?
        Hide
        Filipe Manana added a comment -

        There's no warn log level. Only info, error and debug.

        Show
        Filipe Manana added a comment - There's no warn log level. Only info, error and debug.
        Hide
        Randall Leeds added a comment -

        Right. I'm a space cadet. ERROR it is. Ship it.

        Show
        Randall Leeds added a comment - Right. I'm a space cadet. ERROR it is. Ship it.
        Hide
        Filipe Manana added a comment -

        Fix applied to master, 1.2.x and 1.1.x branches.

        Show
        Filipe Manana added a comment - Fix applied to master, 1.2.x and 1.1.x branches.

          People

          • Assignee:
            Filipe Manana
            Reporter:
            Filipe Manana
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development