Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-5514

Index read repair should use index rpc handlers

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.14.3
    • Fix Version/s: 4.15.0, 5.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Consider base tables T1 and T2, each of which has a global mutable index, call them IDX1 and IDX2. 
      RegionServer A hosts a T1 region and an IDX2 region.
      RegionServer B hosts a T2 region and an IDX1 region. 
       
      Because of prior problems, both IDX1 and IDX2 have lots of unverified rows. Both are under heavy read-load from clients. 
       
      IDX1 coprocs try to scan the T1 region on RS A, but can't because the standard RPC queue is full on RS A because of all the IDX2 clients waiting on cross-server read repairs.
      IDX2 coprocs try to scan the T2 region on RS B, but can't because the standard RPC queue is full on RS B because of the IDX1 clients waiting on cross-server read repairs  . 
       
      It's not a permanent deadlock (eventually we'll start throwing RegionTooBusyExceptions), but it would be unpleasant for clients.
       
      If read-repair used the index RPC pool instead of the standard one, this scenario couldn't happen (unless there were also too many mutable index writes, in which case we're just saturated and back to lots of exceptions.)

        Attachments

        1. PHOENIX-5514.master.001.patch
          2 kB
          Gokcen Iskender
        2. PHOENIX-5514.4.14-HBase-1.3.001.patch
          1 kB
          Kadir OZDEMIR

          Issue Links

            Activity

              People

              • Assignee:
                giskender Gokcen Iskender
                Reporter:
                gjacoby Geoffrey Jacoby
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m