Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-5514

Index read repair should use index rpc handlers

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.14.3
    • 4.15.0, 5.1.0
    • None
    • None

    Description

      Consider base tables T1 and T2, each of which has a global mutable index, call them IDX1 and IDX2. 
      RegionServer A hosts a T1 region and an IDX2 region.
      RegionServer B hosts a T2 region and an IDX1 region. 
       
      Because of prior problems, both IDX1 and IDX2 have lots of unverified rows. Both are under heavy read-load from clients. 
       
      IDX1 coprocs try to scan the T1 region on RS A, but can't because the standard RPC queue is full on RS A because of all the IDX2 clients waiting on cross-server read repairs.
      IDX2 coprocs try to scan the T2 region on RS B, but can't because the standard RPC queue is full on RS B because of the IDX1 clients waiting on cross-server read repairs  . 
       
      It's not a permanent deadlock (eventually we'll start throwing RegionTooBusyExceptions), but it would be unpleasant for clients.
       
      If read-repair used the index RPC pool instead of the standard one, this scenario couldn't happen (unless there were also too many mutable index writes, in which case we're just saturated and back to lots of exceptions.)

      Attachments

        1. PHOENIX-5514.master.001.patch
          2 kB
          Gokcen Iskender
        2. PHOENIX-5514.4.14-HBase-1.3.001.patch
          1 kB
          Kadir OZDEMIR

        Issue Links

          Activity

            People

              giskender Gokcen Iskender
              gjacoby Geoffrey Jacoby
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m