Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: conf, io, ipc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      quote Daryn Sharp from HADOOP-11772:

      CacheBuilder is obscenely expensive for concurrent map, and it requires generating unnecessary garbage even just to look up a key. Replace it with ConcurrentHashMap.

      I identified this issue that impaired my own perf testing under load. The slowdown isn't just the sync. It's the expensive of Connection's ctor stalling other connections. The expensive of ConnectionId#equals causes delays. Synch'ing on connections causes unfair contention unlike a sync'ed method. Concurrency simply hides this.

      BTW, guava Cache is heavyweight. Per local test, ConcurrentHashMap has better overal performance.

        Attachments

        1. HADOOP-12475.01.patch
          5 kB
          Walter Su
        2. HADOOP-12475.02.patch
          5 kB
          Walter Su
        3. HADOOP-12475.03.patch
          5 kB
          Walter Su

          Activity

            People

            • Assignee:
              walter.k.su Walter Su
              Reporter:
              walter.k.su Walter Su
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: