Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-3221

Slow pending replication operations processing

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.8
    • 0.9.9
    • REPL - Replication

    Description

      Currently, we allow up to (2 * number of partitions) concurrent flush operations. However, we have a single replication worker processing the newly generated LSM components. This results in slow pending replication operations processing. We should allow concurrent replication operations processing.

      Attachments

        Activity

          Commit 5f076a8347df9a45634914c76b4117b38b5731a6 in asterixdb's branch refs/heads/master from Murtadha Makki Al Hubail
          [ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=5f076a8347 ]

          ASTERIXDB-3221[REPL] Use IO scheduler for replication ops

          • user model changes: no
          • storage format changes: no
          • interface changes: no

          Details:

          • To allow concurrent replication operations, use the IO scheduler
            to schedule the operations using the exact same scheduling logic
            as the flush operations.
          • Maximum concurrent replication operations = maximum concurrent flush
            operations.
          • Allow connections to replicas to be recycled and reused by the
            replication operations and close them when no more pending operations.
          • Do not halt on replicate operations failures since failures are
            expected and replicas can be re-synced.

          Change-Id: I82aeb60381f90a254ca99274f7e9a38f64bc7a46
          Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17635
          Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>
          Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
          Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

          jira-bot ASF subversion and git services added a comment - Commit 5f076a8347df9a45634914c76b4117b38b5731a6 in asterixdb's branch refs/heads/master from Murtadha Makki Al Hubail [ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=5f076a8347 ] ASTERIXDB-3221 [REPL] Use IO scheduler for replication ops user model changes: no storage format changes: no interface changes: no Details: To allow concurrent replication operations, use the IO scheduler to schedule the operations using the exact same scheduling logic as the flush operations. Maximum concurrent replication operations = maximum concurrent flush operations. Allow connections to replicas to be recycled and reused by the replication operations and close them when no more pending operations. Do not halt on replicate operations failures since failures are expected and replicas can be re-synced. Change-Id: I82aeb60381f90a254ca99274f7e9a38f64bc7a46 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17635 Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

          People

            mhubail Murtadha Makki Al Hubail
            mhubail Murtadha Makki Al Hubail
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: