Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-6621

MapContext.entrySet() slows down when ByteBuffer objects are in the context

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • Release Branch 14.12, 16.11.01
    • 14.12.01
    • framework
    • None
    • Community Day 3 - 2015

    Description

      When MapContext is used anywhere (eg FlexibleStringExpander) and the context contains ByteBuffer objects (either key or value), java will slow down when calling MapContext.entrySet().

      Here is the code in ofbiz, highlighted is the line of code that I believe is the culprit

      MapContext.java
      public Set<Map.Entry<K, V>> entrySet() {
          // walk the stackList and the entries for each Map and if nothing is in for the current key, put it in
          Set<K> resultKeySet = new HashSet<K>();
          culprit --> Set<Map.Entry<K, V>> resultEntrySet = new ListSet<Map.Entry<K, V>>();
          for (Map<K, V> curMap: this.stackList) {
              for (Map.Entry<K, V> curEntry: curMap.entrySet()) {
                  if (!resultKeySet.contains(curEntry.getKey())) {
                      resultKeySet.add(curEntry.getKey());
                      resultEntrySet.add(curEntry);
                  }
              }
          }
          return Collections.unmodifiableSet(resultEntrySet);
      }
      

      Looking at the java api for Map.Entry, the hashCode method for Map.Entry is the result of the hashCode from both key and value. ByteBuffer hash code is dependent upon its content. If a 2mb file is uploaded, the hashCode method
      of ByteBuffer will scan 2mb to generate the hashCode

      ByteBuffer
      http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#hashCode()

      Because buffer hash codes are content-dependent, it is inadvisable to use buffers as keys in hash maps or similar data structures unless it is known that their contents will not change.

      Map.Entry
      http://docs.oracle.com/javase/7/docs/api/java/util/Map.Entry.html#hashCode()

      (e.getKey()==null ? 0 : e.getKey().hashCode()) ^ (e.getValue()==null ? 0 : e.getValue().hashCode())

      HashSet
      http://docs.oracle.com/javase/7/docs/api/java/util/HashSet.html

      This class implements the Set interface, backed by a hash table (actually a HashMap instance

      Example where ByteBuffer objects are created

      org.ofbiz.webapp.event.ServiceEventHandler

      Example stack trace

      "ajp-bio-8009-exec-1894" daemon prio=10 tid=0x00007fa52c070000 nid=0x5c73 runnable [0x00007fa51151b000]
         java.lang.Thread.State: RUNNABLE
      	at java.nio.HeapByteBuffer.ix(HeapByteBuffer.java:131)
      	at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
      	at java.nio.ByteBuffer.hashCode(ByteBuffer.java:1083)
      	at java.util.Objects.hashCode(Objects.java:96)
      	at java.util.HashMap$Entry.hashCode(HashMap.java:847)
      	at java.util.AbstractMap.hashCode(AbstractMap.java:494)
      	at java.util.Objects.hashCode(Objects.java:96)
      	at java.util.HashMap$Entry.hashCode(HashMap.java:847)
      	at java.util.HashMap.hash(HashMap.java:362)
      	at java.util.HashMap.put(HashMap.java:492)
      	at java.util.HashSet.add(HashSet.java:217)
      	at org.ofbiz.base.util.collections.MapContext.entrySet(MapContext.java:306)
      	at java.util.HashMap.putAll(HashMap.java:642)
      	at org.ofbiz.widget.model.ModelFormField$ListOptions.addOptionValues(ModelFormField.java:2716)
      	at org.ofbiz.widget.model.ModelFormField$FieldInfoWithOptions.getAllOptionValues(ModelFormField.java:1985)
      	at org.ofbiz.widget.renderer.macro.MacroFormRenderer.renderDropDownField(MacroFormRenderer.java:747)
      	at org.ofbiz.widget.model.ModelFormField$DropDownField.renderFieldString(ModelFormField.java:1739)
      	at org.ofbiz.widget.model.ModelFormField.renderFieldString(ModelFormField.java:693)
      	at org.ofbiz.widget.renderer.FormRenderer.renderSingleFormString(FormRenderer.java:1195)
      	at org.ofbiz.widget.renderer.FormRenderer.render(FormRenderer.java:261)
      	at org.ofbiz.widget.model.ModelScreenWidget$Form.renderWidgetString(ModelScreenWidget.java:1052)
      	at org.ofbiz.widget.renderer.macro.MacroScreenRenderer.renderScreenletSubWidget(MacroScreenRenderer.java:677)
      	at org.ofbiz.widget.model.ModelScreenWidget$Screenlet.renderWidgetString(ModelScreenWidget.java:598)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreen.renderScreenString(ModelScreen.java:164)
      	at org.ofbiz.widget.model.ScreenFactory.renderReferencedScreen(ScreenFactory.java:211)
      	at org.ofbiz.widget.model.ModelScreenWidget$IncludeScreen.renderWidgetString(ModelScreenWidget.java:779)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Container.renderWidgetString(ModelScreenWidget.java:459)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSection.renderWidgetString(ModelScreenWidget.java:905)
      	at org.ofbiz.widget.model.ModelScreenWidget$SectionsRenderer.render(ModelScreenWidget.java:127)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSectionInclude.renderWidgetString(ModelScreenWidget.java:942)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSection.renderWidgetString(ModelScreenWidget.java:905)
      	at org.ofbiz.widget.model.ModelScreenWidget$SectionsRenderer.render(ModelScreenWidget.java:127)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSectionInclude.renderWidgetString(ModelScreenWidget.java:942)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSection.renderWidgetString(ModelScreenWidget.java:905)
      	at org.ofbiz.widget.model.ModelScreenWidget$SectionsRenderer.render(ModelScreenWidget.java:127)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorSectionInclude.renderWidgetString(ModelScreenWidget.java:942)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Container.renderWidgetString(ModelScreenWidget.java:459)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Container.renderWidgetString(ModelScreenWidget.java:459)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Container.renderWidgetString(ModelScreenWidget.java:459)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreen.renderScreenString(ModelScreen.java:164)
      	at org.ofbiz.widget.model.ScreenFactory.renderReferencedScreen(ScreenFactory.java:211)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorScreen.renderWidgetString(ModelScreenWidget.java:859)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreen.renderScreenString(ModelScreen.java:164)
      	at org.ofbiz.widget.model.ScreenFactory.renderReferencedScreen(ScreenFactory.java:211)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorScreen.renderWidgetString(ModelScreenWidget.java:859)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreen.renderScreenString(ModelScreen.java:164)
      	at org.ofbiz.widget.model.ScreenFactory.renderReferencedScreen(ScreenFactory.java:211)
      	at org.ofbiz.widget.model.ModelScreenWidget$DecoratorScreen.renderWidgetString(ModelScreenWidget.java:859)
      	at org.ofbiz.widget.model.ModelScreenWidget.renderSubWidgetsString(ModelScreenWidget.java:98)
      	at org.ofbiz.widget.model.ModelScreenWidget$Section.renderWidgetString(ModelScreenWidget.java:280)
      	at org.ofbiz.widget.model.ModelScreen.renderScreenString(ModelScreen.java:164)
      	at org.ofbiz.widget.renderer.ScreenRenderer.render(ScreenRenderer.java:136)
      	at org.ofbiz.widget.renderer.ScreenRenderer.render(ScreenRenderer.java:98)
      	at org.ofbiz.widget.renderer.macro.MacroScreenViewHandler.render(MacroScreenViewHandler.java:157)
      	at org.ofbiz.webapp.control.RequestHandler.renderView(RequestHandler.java:989)
      	at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:676)
      	at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:221)
      	at org.ofbiz.webapp.control.ControlServlet.doPost(ControlServlet.java:89)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:321)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
      	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
      	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
      	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
      	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
      	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
      	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
      	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
      	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
      	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
      	- locked <0x00000006f95b1cf0> (a org.apache.tomcat.util.net.SocketWrapper)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:745)
      

      The solution would be not to use HashSet but instead a simpler Set class such as a Set backed by a List.

      Attachments

        1. MapContext-v2.patch
          2 kB
          Gareth Carter
        2. MapContext.patch
          4 kB
          Gareth Carter

        Activity

          People

            mbrohl Michael Brohl
            gareth.carter Gareth Carter
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: