Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-2817

Compaction daemon will only compact views for shards that contain the design doc

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: Database Core
    • Labels:
      None

      Description

      The compaction daemon is currently using couch_db:enum_docs/4 [1] to get the design documents for a database. This is a problem because couch_db:enum_docs/4 operates on a shard - we will therefore only get a list of all the design documents on that shard, rather than all design documents for the DB.

      This means view shards for the DB shard will not get compacted unless the design doc happens to be on that shard.

      I tested this out by generating some fragmented views and running the compaction daemon. After completion I found that only one view shard had been compacted [2] and verified that it was the shard that contains the design doc [3].

      [1] https://github.com/apache/couchdb-couch/blob/master/src/couch_compaction_daemon.erl#L235

      [2] https://gist.github.com/mikewallace1979/abf87e1ece983fad85b3#file-view_shard_sizes-sh-L5

      [3] https://gist.github.com/mikewallace1979/abf87e1ece983fad85b3#file-ddoc-sh-L1-L2

        Activity

        Hide
        mikewallace Mike Wallace added a comment -

        I hoped there'd be a simple fix here using mem3:dname/1 to get the cluster-level DB name and fabric:design_docs/1 to get the design docs, but this then fails for things like the _nodes and _dbs DBs which do not exist at the cluster level.

        Show
        mikewallace Mike Wallace added a comment - I hoped there'd be a simple fix here using mem3:dname/1 to get the cluster-level DB name and fabric:design_docs/1 to get the design docs, but this then fails for things like the _nodes and _dbs DBs which do not exist at the cluster level.
        Hide
        mikewallace Mike Wallace added a comment -

        Noting there is a similar problem when attempting to compact view shards manually: http://stackoverflow.com/questions/35730682/couchdb-2-0-view-compaction

        Show
        mikewallace Mike Wallace added a comment - Noting there is a similar problem when attempting to compact view shards manually: http://stackoverflow.com/questions/35730682/couchdb-2-0-view-compaction

          People

          • Assignee:
            Unassigned
            Reporter:
            mikewallace Mike Wallace
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development