CouchDB
  1. CouchDB
  2. COUCHDB-1252

A way to have views return _deleted documents

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.3, 1.1
    • Fix Version/s: None
    • Component/s: JavaScript View Server
    • Labels:
      None
    • Skill Level:
      New Contributors Level (Easy)

      Description

      Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.

      I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.

        Activity

        Hide
        James Howe added a comment - - edited

        Any sign of this or shall we just do it ourselves (and due to contractual reasons won't be able to CLA it)?

        Show
        James Howe added a comment - - edited Any sign of this or shall we just do it ourselves (and due to contractual reasons won't be able to CLA it)?
        Hide
        Jason Smith added a comment -

        Hi, James. There is an Iris Couch sprint starting Thursday September 8, running for one week. I expect to complete it during the sprint.

        Show
        Jason Smith added a comment - Hi, James. There is an Iris Couch sprint starting Thursday September 8, running for one week. I expect to complete it during the sprint.
        Hide
        James Howe added a comment -

        ETA on this patch Jason?

        Show
        James Howe added a comment - ETA on this patch Jason?
        Hide
        Robert Newson added a comment -

        Same level as include_design, which is per ddoc.

        Show
        Robert Newson added a comment - Same level as include_design, which is per ddoc.
        Hide
        Jason Smith added a comment -

        To Damien, I say: well, why can't I see deleted docs in a view? Why do I get a 404 when I query for them? Because they are deleted. Damien's code contradicts Damien's words, and the code makes better sense. To expose _deleted documents everywhere is a leaky abstraction.

        However, in the spirit of "design by committee is bad" and to learn a different part of CouchDB, I would like to write this patch. Paul, should it be a design option or a view option?

        For example:

        { "_id": "_design/example"
        , "options":

        { "these_are":"options global to the ddoc", "local_seq":true, "include_deleted":true}

        , "views":
        { "deleted_docs_A":
        { "options":

        {"these_are":"local to this view", "include_design":true, "include_deleted":true}

        , "map":"function(D)

        { if(D._deleted) emit(D._id, 1) }

        "
        }
        , "deleted_docs_B":
        { "options":

        {"not using include_deleted": true}

        , "map":"function(D)

        { if(D._deleted) emit(D._id, 1) /* Does it emit? */ }

        }
        }
        }

        With design options, deleted_docs_B will emit, but using view options, it won't. I prefer it to be per-view. In fact I did not even realize design options existed until I checked the source.

        Thoughts? Thanks.

        Show
        Jason Smith added a comment - To Damien, I say: well, why can't I see deleted docs in a view? Why do I get a 404 when I query for them? Because they are deleted. Damien's code contradicts Damien's words, and the code makes better sense. To expose _deleted documents everywhere is a leaky abstraction. However, in the spirit of "design by committee is bad" and to learn a different part of CouchDB, I would like to write this patch. Paul, should it be a design option or a view option? For example: { "_id": "_design/example" , "options": { "these_are":"options global to the ddoc", "local_seq":true, "include_deleted":true} , "views": { "deleted_docs_A": { "options": {"these_are":"local to this view", "include_design":true, "include_deleted":true} , "map":"function(D) { if(D._deleted) emit(D._id, 1) } " } , "deleted_docs_B": { "options": {"not using include_deleted": true} , "map":"function(D) { if(D._deleted) emit(D._id, 1) /* Does it emit? */ } } } } With design options, deleted_docs_B will emit, but using view options, it won't. I prefer it to be per-view. In fact I did not even realize design options existed until I checked the source. Thoughts? Thanks.
        Hide
        Paul Joseph Davis added a comment -

        @Jason,

        A couple weeks ago I would have agreed completely with you. So much in fact that when someone reported a bug that _bulk_docs was storing the body of docs submitted with _deleted: true set, it was immediately assumed to be a bug and was promptly fixed.

        Until Damien said "That was on purpose so people can say why something was deleted."

        In reality, the only "special" sauce for deleted docs is that they're not considered conflicts. Other than that, you can still create a doc based on the revision of a deleted doc. In fact you can rescue an edit history from any known revision. But I digress.

        Either way, "auditing information about deleted docs" is apparently precisely what deleted doc bodies were for, so I don't see a reason to not have an option to see them.

        Show
        Paul Joseph Davis added a comment - @Jason, A couple weeks ago I would have agreed completely with you. So much in fact that when someone reported a bug that _bulk_docs was storing the body of docs submitted with _deleted: true set, it was immediately assumed to be a bug and was promptly fixed. Until Damien said "That was on purpose so people can say why something was deleted." In reality, the only "special" sauce for deleted docs is that they're not considered conflicts. Other than that, you can still create a doc based on the revision of a deleted doc. In fact you can rescue an edit history from any known revision. But I digress. Either way, "auditing information about deleted docs" is apparently precisely what deleted doc bodies were for, so I don't see a reason to not have an option to see them.
        Hide
        Jason Smith added a comment -

        I propose this issue be closed as NOFIX (or whatever it's called). Allowing include_deleted would confuse the concept of "deleted" in the way that calling them "revisions" implies that CouchDB revisions are great for a wiki.

        James, your requirements are brilliant. That sounds like a great app. But IMO if you need data to remain indefinitely (for audit), then you should not be deleting documents and then exploiting the fact that CouchDB keeps deleted documents for revision tracking. In your application, "deleted" means "invisible to users but the auditors still want to know about it" so that is an application-level detail. That is exactly what views are for.

        For an application, _deleted means it's dead. Advanced tools like the replicator must follow the doc into the afterlife but that's not an application's concern.

        Show
        Jason Smith added a comment - I propose this issue be closed as NOFIX (or whatever it's called). Allowing include_deleted would confuse the concept of "deleted" in the way that calling them "revisions" implies that CouchDB revisions are great for a wiki. James, your requirements are brilliant. That sounds like a great app. But IMO if you need data to remain indefinitely (for audit), then you should not be deleting documents and then exploiting the fact that CouchDB keeps deleted documents for revision tracking. In your application, "deleted" means "invisible to users but the auditors still want to know about it" so that is an application-level detail. That is exactly what views are for. For an application, _deleted means it's dead. Advanced tools like the replicator must follow the doc into the afterlife but that's not an application's concern.
        Hide
        Paul Joseph Davis added a comment -

        We could fairly easily add a include_deleted option to the view options parameter the same we do for include_design.

        Show
        Paul Joseph Davis added a comment - We could fairly easily add a include_deleted option to the view options parameter the same we do for include_design.

          People

          • Assignee:
            Unassigned
            Reporter:
            James Howe
          • Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development