Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-3337

End-point _local_docs doesn't conform to query params of _all_docs

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      There are following issues with /db/_local_doc end-point

      1. It ignores include_docs parameter
      2. It reports incorrect total_rows value, returning total rows of non-local docs instead
      3. It crashes with badrecord full_doc_info when query trying to skip records

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user eiri opened a pull request:

          https://github.com/apache/couchdb-couch-mrview/pull/72

          Fix end-point _local_docs

          This fixes the following issues with `/

          {db}

          /_local_docs` end-point:

          1. Query ignores `include_docs` parameter
          1. Query reports incorrect `total_rows` value, returning total rows for non-local docs instead
          1. Query crashes with `badrecord` when trying to skip records, e.g. when `start_key` not the first local doc

          COUCHDB-3337

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/cloudant/couchdb-couch-mrview 84617-fix-_local_docs

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/couchdb-couch-mrview/pull/72.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #72


          commit 021e46ad2792460e968021de84d42a8f3faa5f4d
          Author: Eric Avdey <eiri@eiri.ca>
          Date: 2017-03-17T14:04:53Z

          Make _local_docs conform to include_docs

          commit 6867449cce3618a065f8c966c65a391d42e35283
          Author: Eric Avdey <eiri@eiri.ca>
          Date: 2017-03-21T15:03:09Z

          Adopt all_docs_reduce_to_count/1 to _local docs

          We are using all_docs_reduce_to_count/1 in
          _local_docs handler, but reductions got
          from local btree are different from
          reductions passed from id btree's enumerator.

          This change converts passed local's KVs to
          list of expected #doc_full_info records.

          commit 79ad8bd1bf52c363334a5e55c627f46109d494bf
          Author: Eric Avdey <eiri@eiri.ca>
          Date: 2017-03-21T19:49:24Z

          Fix total_row for _local_docs

          commit beaa1bab96c8876dce1e49a91a59041556c789ec
          Author: Eric Avdey <eiri@eiri.ca>
          Date: 2017-03-21T22:37:08Z

          Add utests for _local_docs


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user eiri opened a pull request: https://github.com/apache/couchdb-couch-mrview/pull/72 Fix end-point _local_docs This fixes the following issues with `/ {db} /_local_docs` end-point: 1. Query ignores `include_docs` parameter 1. Query reports incorrect `total_rows` value, returning total rows for non-local docs instead 1. Query crashes with `badrecord` when trying to skip records, e.g. when `start_key` not the first local doc COUCHDB-3337 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloudant/couchdb-couch-mrview 84617-fix-_local_docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/couchdb-couch-mrview/pull/72.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #72 commit 021e46ad2792460e968021de84d42a8f3faa5f4d Author: Eric Avdey <eiri@eiri.ca> Date: 2017-03-17T14:04:53Z Make _local_docs conform to include_docs commit 6867449cce3618a065f8c966c65a391d42e35283 Author: Eric Avdey <eiri@eiri.ca> Date: 2017-03-21T15:03:09Z Adopt all_docs_reduce_to_count/1 to _local docs We are using all_docs_reduce_to_count/1 in _local_docs handler, but reductions got from local btree are different from reductions passed from id btree's enumerator. This change converts passed local's KVs to list of expected #doc_full_info records. commit 79ad8bd1bf52c363334a5e55c627f46109d494bf Author: Eric Avdey <eiri@eiri.ca> Date: 2017-03-21T19:49:24Z Fix total_row for _local_docs commit beaa1bab96c8876dce1e49a91a59041556c789ec Author: Eric Avdey <eiri@eiri.ca> Date: 2017-03-21T22:37:08Z Add utests for _local_docs
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user eiri closed the pull request at:

          https://github.com/apache/couchdb-couch-mrview/pull/72

          Show
          githubbot ASF GitHub Bot added a comment - Github user eiri closed the pull request at: https://github.com/apache/couchdb-couch-mrview/pull/72
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 860f23cb57e0af84b668609676ba93c5edb769d5 in couchdb's branch refs/heads/master from Eric Avdey
          [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=860f23c ]

          Fix _local_docs end-point

          This is a second attempt to fix _local_docs end-point. The previous one didn't
          work on big enough btree_local, because local btree doesn't have reduction fun,
          so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match
          as soon as btree_local was getting kp_node. Also using full fold to calculate
          total_rows value turned out to be resource expensive when a database have
          significant number of local documents.

          This fix avoids calculating of total_rows and offset instead always setting
          them to null and also setting to null update_seq when requested, since it
          doesn't have meaning in context of local documents.

          A fabric module fabric_view_all_docs.erl was copied and modified as
          fabric_view_local_docs.erl, because re-using it for processing of both types of
          the documents was getting rather convoluted.

          Jira: COUCHDB-3337

          Show
          jira-bot ASF subversion and git services added a comment - Commit 860f23cb57e0af84b668609676ba93c5edb769d5 in couchdb's branch refs/heads/master from Eric Avdey [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=860f23c ] Fix _local_docs end-point This is a second attempt to fix _local_docs end-point. The previous one didn't work on big enough btree_local, because local btree doesn't have reduction fun, so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match as soon as btree_local was getting kp_node. Also using full fold to calculate total_rows value turned out to be resource expensive when a database have significant number of local documents. This fix avoids calculating of total_rows and offset instead always setting them to null and also setting to null update_seq when requested, since it doesn't have meaning in context of local documents. A fabric module fabric_view_all_docs.erl was copied and modified as fabric_view_local_docs.erl, because re-using it for processing of both types of the documents was getting rather convoluted. Jira: COUCHDB-3337
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit c40f66d06bafbeb2e04c989e1a3ecfe5c034fde4 in couchdb's branch refs/heads/jenkins-packages-2 from Eric Avdey
          [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=c40f66d0 ]

          Fix _local_docs end-point

          This is a second attempt to fix _local_docs end-point. The previous one didn't
          work on big enough btree_local, because local btree doesn't have reduction fun,
          so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match
          as soon as btree_local was getting kp_node. Also using full fold to calculate
          total_rows value turned out to be resource expensive when a database have
          significant number of local documents.

          This fix avoids calculating of total_rows and offset instead always setting
          them to null and also setting to null update_seq when requested, since it
          doesn't have meaning in context of local documents.

          A fabric module fabric_view_all_docs.erl was copied and modified as
          fabric_view_local_docs.erl, because re-using it for processing of both types of
          the documents was getting rather convoluted.

          Jira: COUCHDB-3337

          Show
          jira-bot ASF subversion and git services added a comment - Commit c40f66d06bafbeb2e04c989e1a3ecfe5c034fde4 in couchdb's branch refs/heads/jenkins-packages-2 from Eric Avdey [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=c40f66d0 ] Fix _local_docs end-point This is a second attempt to fix _local_docs end-point. The previous one didn't work on big enough btree_local, because local btree doesn't have reduction fun, so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match as soon as btree_local was getting kp_node. Also using full fold to calculate total_rows value turned out to be resource expensive when a database have significant number of local documents. This fix avoids calculating of total_rows and offset instead always setting them to null and also setting to null update_seq when requested, since it doesn't have meaning in context of local documents. A fabric module fabric_view_all_docs.erl was copied and modified as fabric_view_local_docs.erl, because re-using it for processing of both types of the documents was getting rather convoluted. Jira: COUCHDB-3337
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 860f23cb57e0af84b668609676ba93c5edb769d5 in couchdb's branch refs/heads/680-fix-couchdb-1283-test from Eric Avdey
          [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=860f23c ]

          Fix _local_docs end-point

          This is a second attempt to fix _local_docs end-point. The previous one didn't
          work on big enough btree_local, because local btree doesn't have reduction fun,
          so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match
          as soon as btree_local was getting kp_node. Also using full fold to calculate
          total_rows value turned out to be resource expensive when a database have
          significant number of local documents.

          This fix avoids calculating of total_rows and offset instead always setting
          them to null and also setting to null update_seq when requested, since it
          doesn't have meaning in context of local documents.

          A fabric module fabric_view_all_docs.erl was copied and modified as
          fabric_view_local_docs.erl, because re-using it for processing of both types of
          the documents was getting rather convoluted.

          Jira: COUCHDB-3337

          Show
          jira-bot ASF subversion and git services added a comment - Commit 860f23cb57e0af84b668609676ba93c5edb769d5 in couchdb's branch refs/heads/680-fix-couchdb-1283-test from Eric Avdey [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=860f23c ] Fix _local_docs end-point This is a second attempt to fix _local_docs end-point. The previous one didn't work on big enough btree_local, because local btree doesn't have reduction fun, so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match as soon as btree_local was getting kp_node. Also using full fold to calculate total_rows value turned out to be resource expensive when a database have significant number of local documents. This fix avoids calculating of total_rows and offset instead always setting them to null and also setting to null update_seq when requested, since it doesn't have meaning in context of local documents. A fabric module fabric_view_all_docs.erl was copied and modified as fabric_view_local_docs.erl, because re-using it for processing of both types of the documents was getting rather convoluted. Jira: COUCHDB-3337
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 825d0add611289eb624df5ef469a86c8ae3996b0 in couchdb's branch refs/heads/jenkins-packages-2 from Eric Avdey
          [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=825d0ad ]

          Fix _local_docs end-point

          This is a second attempt to fix _local_docs end-point. The previous one didn't
          work on big enough btree_local, because local btree doesn't have reduction fun,
          so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match
          as soon as btree_local was getting kp_node. Also using full fold to calculate
          total_rows value turned out to be resource expensive when a database have
          significant number of local documents.

          This fix avoids calculating of total_rows and offset instead always setting
          them to null and also setting to null update_seq when requested, since it
          doesn't have meaning in context of local documents.

          A fabric module fabric_view_all_docs.erl was copied and modified as
          fabric_view_local_docs.erl, because re-using it for processing of both types of
          the documents was getting rather convoluted.

          Jira: COUCHDB-3337

          Show
          jira-bot ASF subversion and git services added a comment - Commit 825d0add611289eb624df5ef469a86c8ae3996b0 in couchdb's branch refs/heads/jenkins-packages-2 from Eric Avdey [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=825d0ad ] Fix _local_docs end-point This is a second attempt to fix _local_docs end-point. The previous one didn't work on big enough btree_local, because local btree doesn't have reduction fun, so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match as soon as btree_local was getting kp_node. Also using full fold to calculate total_rows value turned out to be resource expensive when a database have significant number of local documents. This fix avoids calculating of total_rows and offset instead always setting them to null and also setting to null update_seq when requested, since it doesn't have meaning in context of local documents. A fabric module fabric_view_all_docs.erl was copied and modified as fabric_view_local_docs.erl, because re-using it for processing of both types of the documents was getting rather convoluted. Jira: COUCHDB-3337

            People

            • Assignee:
              eiri Eric Avdey
              Reporter:
              eiri Eric Avdey
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development