Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-1755

BatchWriter blocks all addMutation calls while binning mutations

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.6, 1.7.2, 1.8.0
    • Component/s: client
    • Labels:
      None

      Description

      Through code inspection, we found that the BatchWriter bins mutations inside of a synchronized block that covers calls to addMutation. Binning potentially involves lookups of tablet metadata and processes a fair amount of information. We will get better parallelism if we can either unlock the lock while binning, dedicate another thread to do the binning, or use one of the send threads to do the binning.

      This has not been verified empirically yet, so there is not yet any profiling info to indicate the level of improvement that we should expect. Profiling and repeatable demonstration of this performance bottleneck should be the first step on this ticket.

        Attachments

        1. TSBWBinningSimulator.java
          3 kB
          Adam Fuchs
        2. Screen Shot 2016-03-08 at 9.47.39 AM.png
          73 kB
          Adam Fuchs
        3. ACCUMULO-1755.patch
          16 kB
          Dave Marion
        4. 1755-perf-test.patch
          5 kB
          Dave Marion
        5. 1755-nosync-perf-test.patch
          27 kB
          Dave Marion

          Issue Links

            Activity

              People

              • Assignee:
                dlmarion Dave Marion
                Reporter:
                afuchs Adam Fuchs
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h