HBase
  1. HBase
  2. HBASE-6651

Improve thread safety of HTablePool

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.94.1
    • Fix Version/s: None
    • Component/s: Client
    • Labels:
      None

      Description

      There are some operations in HTablePool accessing PoolMap in multiple places without any explicit synchronization.

      For example HTablePool.closeTablePool() calls PoolMap.values(), and calls PoolMap.remove(). If other threads add new instances to the pool in the middle of the calls, the newly added instances might be dropped. (HTablePool.closeTablePool() also has another problem that calling it by multiple threads causes accessing HTable by multiple threads.)

      Moreover, PoolMap is not thread safe for the same reason.

      For example PoolMap.put() calles ConcurrentMap.get() and calles ConcurrentMap.put(). If other threads add a new instance to the concurent map in the middle of the calls, the new instance might be dropped.

      And also implementations of Pool have the same problems.

      1. HBASE-6651.patch
        73 kB
        Hiroshi Ikeda
      2. HBASE-6651-V10.patch
        108 kB
        Hiroshi Ikeda
      3. HBASE-6651-V11.patch
        109 kB
        Hiroshi Ikeda
      4. HBASE-6651-V12.patch
        117 kB
        Hiroshi Ikeda
      5. HBASE-6651-V13.patch
        130 kB
        Hiroshi Ikeda
      6. HBASE-6651-V14-0.96.patch
        85 kB
        Hiroshi Ikeda
      7. HBASE-6651-V14-0.98.patch
        84 kB
        Hiroshi Ikeda
      8. HBASE-6651-V14-trunk.patch
        83 kB
        Hiroshi Ikeda
      9. HBASE-6651-V15-0.96.patch
        85 kB
        Hiroshi Ikeda
      10. HBASE-6651-V15-0.98.patch
        85 kB
        Hiroshi Ikeda
      11. HBASE-6651-V15-trunk.patch
        83 kB
        Hiroshi Ikeda
      12. HBASE-6651-V16-0.96.patch
        90 kB
        Hiroshi Ikeda
      13. HBASE-6651-V16-0.98.patch
        90 kB
        Hiroshi Ikeda
      14. HBASE-6651-V16-trunk.patch
        89 kB
        Hiroshi Ikeda
      15. HBASE-6651-V2.patch
        78 kB
        Hiroshi Ikeda
      16. HBASE-6651-V3.patch
        81 kB
        Ted Yu
      17. HBASE-6651-V4.patch
        83 kB
        Hiroshi Ikeda
      18. HBASE-6651-V5.patch
        83 kB
        Hiroshi Ikeda
      19. HBASE-6651-V6.patch
        86 kB
        Hiroshi Ikeda
      20. HBASE-6651-V7.patch
        87 kB
        Hiroshi Ikeda
      21. HBASE-6651-V8.patch
        95 kB
        Hiroshi Ikeda
      22. HBASE-6651-V9.patch
        103 kB
        Hiroshi Ikeda
      23. sample.zip
        7 kB
        Hiroshi Ikeda
      24. sample.zip
        7 kB
        Hiroshi Ikeda
      25. sharedmap_for_hbaseclient.zip
        6 kB
        Hiroshi Ikeda

        Issue Links

          Activity

          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V16-0.96.patch [ 12637770 ]
          Attachment HBASE-6651-V16-0.98.patch [ 12637771 ]
          Attachment HBASE-6651-V16-trunk.patch [ 12637772 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V15-0.96.patch [ 12637340 ]
          Attachment HBASE-6651-V15-0.98.patch [ 12637341 ]
          Attachment HBASE-6651-V15-trunk.patch [ 12637342 ]
          Hiroshi Ikeda made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V14-0.96.patch [ 12637317 ]
          Attachment HBASE-6651-V14-0.98.patch [ 12637318 ]
          Attachment HBASE-6651-V14-trunk.patch [ 12637319 ]
          stack made changes -
          Fix Version/s 0.95.1 [ 12324288 ]
          stack made changes -
          Fix Version/s 0.95.1 [ 12324288 ]
          Fix Version/s 0.95.0 [ 12324094 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V13.patch [ 12576494 ]
          stack made changes -
          Fix Version/s 0.95.0 [ 12324094 ]
          Fix Version/s 0.96.0 [ 12320040 ]
          Lars Hofhansl made changes -
          Link This issue relates to HBASE-7463 [ HBASE-7463 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V12.patch [ 12562044 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V11.patch [ 12561665 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V10.patch [ 12560716 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V9.patch [ 12560501 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V8.patch [ 12560157 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V7.patch [ 12554470 ]
          Ted Yu made changes -
          Description There are some operations in HTablePool to access to PoolMap in multiple times without any explict synchronization.

          For example HTablePool.closeTablePool() calles PoolMap.values(), and calles PoolMap.remove(). If other threads add new instances to the pool in the middle of the calls, the new added instances might be dropped. (HTablePool.closeTablePool() also has another problem that calling it by multple threads causes accessing HTable by multiple threads.)

          Moreover, PoolMap is not thread safe for the same reason.

          For example PoolMap.put() calles ConcurrentMap.get() and calles ConcurrentMap.put(). If other threads add a new instance to the concurent map in the middle of the calls, the new instance might be dropped.

          And also implementations of Pool have the same problems.
          There are some operations in HTablePool accessing PoolMap in multiple places without any explicit synchronization.

          For example HTablePool.closeTablePool() calls PoolMap.values(), and calls PoolMap.remove(). If other threads add new instances to the pool in the middle of the calls, the newly added instances might be dropped. (HTablePool.closeTablePool() also has another problem that calling it by multiple threads causes accessing HTable by multiple threads.)

          Moreover, PoolMap is not thread safe for the same reason.

          For example PoolMap.put() calles ConcurrentMap.get() and calles ConcurrentMap.put(). If other threads add a new instance to the concurent map in the middle of the calls, the new instance might be dropped.

          And also implementations of Pool have the same problems.
          Ted Yu made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          Ted Yu made changes -
          Link This issue is related to HBASE-6956 [ HBASE-6956 ]
          Ted Yu made changes -
          Assignee Hiroshi Ikeda [ ikeda ]
          Ted Yu made changes -
          Summary Thread safety of HTablePool is doubtful Improve thread safety of HTablePool
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.96.0 [ 12320040 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V6.patch [ 12550936 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V5.patch [ 12550927 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V4.patch [ 12550744 ]
          Ted Yu made changes -
          Attachment HBASE-6651-V3.patch [ 12550486 ]
          Ted Yu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651-V2.patch [ 12549457 ]
          Hiroshi Ikeda made changes -
          Attachment HBASE-6651.patch [ 12548722 ]
          Hiroshi Ikeda made changes -
          Attachment sharedmap_for_hbaseclient.zip [ 12546064 ]
          Hiroshi Ikeda made changes -
          Attachment sample.zip [ 12545833 ]
          Hiroshi Ikeda made changes -
          Field Original Value New Value
          Attachment sample.zip [ 12545703 ]
          Hiroshi Ikeda created issue -

            People

            • Assignee:
              Hiroshi Ikeda
              Reporter:
              Hiroshi Ikeda
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:

                Development