Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.1, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In the master, we will do the full scan of META in some situations
      for example,
      1.master start up
      2.CatalogJanitor do the full scan per 5 mins
      3.ServerShutdownHandler, getServerUserRegions for dead server.

      For the online applications, we should try the best to reduce the process time of ServerShutdownHandler in the situation 3.
      However, we found MetaReader#getServerUserRegions take 14mins for 10w regions in our production environment.
      And it is caused by two reasons:
      The first, we don't use cache and get one row per next() when fully scan .META.
      The second, "hbase.ipc.client.tcpnodelay" is false as default, and in our environment it take 40ms for per next() (It is related to the length of row in the .META. , if someone also found, could try to set it true)

      For this issue, I think we could set the caching when do the full scan of META

      1. 5913-v2.txt
        0.8 kB
        Ted Yu
      2. HBASE-5913.patch
        0.8 kB
        chunhui shen

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            chunhui shen
            Reporter:
            chunhui shen
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development