Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6968

Several HBase write perf improvement

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.90.6, 0.92.2, 0.94.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Here are 2 hbase write performance improvements recently:

      1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is not cheap operation.
      In HLog.java:
      orig:

      HLog.java
        newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
      

      new:

        newWriter = createWriter(fs, newPath, conf);
      

      2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove the synchronization overhead in the normal case.
      orig:

      HBaseRpcMetrics.java
        public synchronized void inc(String name, int amt) {	
          MetricsTimeVaryingRate m = get(name);	
          if (m == null) {	
            m = create(name);	
          }	
          m.inc(amt);	
        }
      

      new:

        public void inc(String name, int amt) {	
          MetricsTimeVaryingRate m = get(name);	
          if (m == null) {	
            synchronized (this) {	
              if ((m = get(name)) == null) {	
                m = create(name);	
              }	
            }	
          }	
          m.inc(amt);	
        }
      

      =====================
      orig:

      MemStoreFlusher.java
        public synchronized void reclaimMemStoreMemory() {	
          if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
            flushSomeRegions();	
          }
        }	
      

      new:

        public void reclaimMemStoreMemory() {	
          if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
            flushSomeRegions();	
          }
        }	
        private synchronized void flushSomeRegions() {	
          if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {	
            return; // double check the global memstore size inside of the synchronized block.	
          }	
       ...   
       }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              liyin Liyin Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: