Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, hbase-10070
    • Component/s: None
    • Labels:
      None

      Description

      During testing with the IT in HBASE-10818, found an issue to do with how "close" of scanners was handled.

      1. 11214-1.txt
        19 kB
        Devaraj Das
      2. 11214-2.txt
        13 kB
        Devaraj Das
      3. 0040-HBASE-11214.-Fixes-for-scans-on-a-replicated-table.patch
        22 kB
        Enis Soztutar

        Activity

        Hide
        Devaraj Das added a comment -

        This introduces another pool for handling "close" of scanners. Earlier the close calls would be in the same pool and would cause issues when tasks in the pool are cancelled (we don't want to actually cancel the tasks scheduled for doing "close"). We'd also need to be aware of what type of tasks complete when a task in the pool completes and handle it accordingly... Having a separate pool simplifies handling these issues.
        I also removed some unused constructors in ClientScanner/SmallScanner/ReverseScanner and changed the annotation to Private on those classes.

        Show
        Devaraj Das added a comment - This introduces another pool for handling "close" of scanners. Earlier the close calls would be in the same pool and would cause issues when tasks in the pool are cancelled (we don't want to actually cancel the tasks scheduled for doing "close"). We'd also need to be aware of what type of tasks complete when a task in the pool completes and handle it accordingly... Having a separate pool simplifies handling these issues. I also removed some unused constructors in ClientScanner/SmallScanner/ReverseScanner and changed the annotation to Private on those classes.
        Hide
        Enis Soztutar added a comment -

        I think you don't need a separate thread pool. Also maybe not even a separate completion service. We don't close the pool, only cancel the threads in the completion service. Can we just submit the close requests to the pool rather than CompetionService?

        Show
        Enis Soztutar added a comment - I think you don't need a separate thread pool. Also maybe not even a separate completion service. We don't close the pool, only cancel the threads in the completion service. Can we just submit the close requests to the pool rather than CompetionService?
        Hide
        Devaraj Das added a comment -

        Yes, we could use the same pool. The issue is, as I said before, we would then need to keep a track of which task completed because in that case, we would have two types of tasks in the pool - one the "scan" and another "close". We would have to pass in more information etc. when we schedule a task... Also when a "scan" task completes, we cancel the other redundant scan tasks. This would also need to be aware that it shouldn't cancel the "close" tasks.
        In summary, it's doable very much but it's more code... Having two pools doesn't harm anything and these are internal implementation classes, not user-visible..

        Show
        Devaraj Das added a comment - Yes, we could use the same pool. The issue is, as I said before, we would then need to keep a track of which task completed because in that case, we would have two types of tasks in the pool - one the "scan" and another "close". We would have to pass in more information etc. when we schedule a task... Also when a "scan" task completes, we cancel the other redundant scan tasks. This would also need to be aware that it shouldn't cancel the "close" tasks. In summary, it's doable very much but it's more code... Having two pools doesn't harm anything and these are internal implementation classes, not user-visible..
        Hide
        Devaraj Das added a comment -

        But let me check if the CompletionService can share a pool. If it can, I can have one pool (and two completion services).

        Show
        Devaraj Das added a comment - But let me check if the CompletionService can share a pool. If it can, I can have one pool (and two completion services).
        Hide
        Devaraj Das added a comment -

        Never mind my previous comment on CompletionService/pool. It was my oversight on pool's functionality. This patch should address your feedback, Enis Soztutar.

        Show
        Devaraj Das added a comment - Never mind my previous comment on CompletionService/pool. It was my oversight on pool's functionality. This patch should address your feedback, Enis Soztutar .
        Hide
        Enis Soztutar added a comment -

        Thanks Devaraj. +1 for the patch.

        Show
        Enis Soztutar added a comment - Thanks Devaraj. +1 for the patch.
        Hide
        Devaraj Das added a comment -

        Committed to hbase-10070 branch. Thanks for the review, Enis.

        Show
        Devaraj Das added a comment - Committed to hbase-10070 branch. Thanks for the review, Enis.
        Hide
        Enis Soztutar added a comment -

        Attaching rebased patch for master that is committed

        Show
        Enis Soztutar added a comment - Attaching rebased patch for master that is committed
        Hide
        Enis Soztutar added a comment -

        Committed to master as part of hbase-10070 branch merge

        Show
        Enis Soztutar added a comment - Committed to master as part of hbase-10070 branch merge
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5245 (See https://builds.apache.org/job/HBase-TRUNK/5245/)
        HBASE-11214. Fixes for scans on a replicated table (enis: rev 1daf8acfc9f48551a4d427bf71fb32f56371391b)

        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallReversedScanner.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5245 (See https://builds.apache.org/job/HBase-TRUNK/5245/ ) HBASE-11214 . Fixes for scans on a replicated table (enis: rev 1daf8acfc9f48551a4d427bf71fb32f56371391b) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallReversedScanner.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development