Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.0.0-alpha-1, 2.0.0
    • Fix Version/s: 2.0.0-alpha-2, 2.0.0
    • Component/s: Scanners
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Found this while debugging HBASE-18186. When we try to reopen the scanners on the storefiles while trying to switch over from pread to stream, we do not use the HStore's reentrant lock to get the current Storefiles from the StoreFileManager. All the scan APIs are guarded under that and we must do it here also other wise the CompactedHfileDischarger may cause race issues with the HStore's datastructures like here

      2017-06-14 18:16:17,223 WARN  [RpcServer.default.FPBQ.Fifo.handler=23,queue=1,port=16020] regionserver.StoreScanner: failed to switch to stream read
      java.lang.NullPointerException
              at org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(StoreFileScanner.java:133)
              at org.apache.hadoop.hbase.regionserver.HStore.getScanners(HStore.java:1221)
              at org.apache.hadoop.hbase.regionserver.StoreScanner.trySwitchToStreamRead(StoreScanner.java:997)
              at org.apache.hadoop.hbase.regionserver.StoreScanner.shipped(StoreScanner.java:1134)
              at org.apache.hadoop.hbase.regionserver.KeyValueHeap.shipped(KeyValueHeap.java:445)
              at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.shipped(HRegion.java:6459)
              at org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerShippedCallBack.run(RSRpcServices.java:339)
              at org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:252)
              at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:166)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
              at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
      

      I have a working patch fixing this problem. Will do some more testing and try to upload the patch after I write a test case for this.

        Attachments

        1. HBASE-18221_2_fortestcasefailure.patch
          26 kB
          ramkrishna.s.vasudevan
        2. HBASE-18221_2.patch
          21 kB
          ramkrishna.s.vasudevan
        3. HBASE-18221_2.patch
          22 kB
          ramkrishna.s.vasudevan

          Activity

            People

            • Assignee:
              ram_krish ramkrishna.s.vasudevan
              Reporter:
              ram_krish ramkrishna.s.vasudevan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: