OpenJPA
  1. OpenJPA
  2. OPENJPA-2220

Persistent field fetching statistic tool

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.1, 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: tooling
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      This task is to create a statistic tool which monitors the persistent fields fetching and finds out the fields which are never accessed. Based on the statistic data, user can set the field access type to LAZY to improve the performance by eliminating the data loading and processing time.

      1. READMEpatch
        3 kB
        Helen Xu
      2. statisticToolDocPatch.txt
        6 kB
        Helen Xu
      3. OPENJPA-2220Patch.txt
        43 kB
        Helen Xu

        Activity

        Hide
        Pinaki Poddar added a comment -

        I think it is a good/useful feature. And I appreciate your contribution.
        I would suggest to expand the scope of this idea to collect more statistics on field access patterns. It has been a long-held wish for many original OpenJPA developers to collect such detailed statistics, so that based on the access pattern of correlated fields we can prescribe which fetch groups will best serve an applications need.

        A nit: to determine whether a field is persistent, the patch code checks AccessCode and if the bean-method starts with "getX" etc. That check should include boolean fields which may start with "is". You should find the logic somewhere (do not remember exactly where, but around AccessCode or classes that use it), some code that determines whether a named field is persistent.

        Show
        Pinaki Poddar added a comment - I think it is a good/useful feature. And I appreciate your contribution. I would suggest to expand the scope of this idea to collect more statistics on field access patterns. It has been a long-held wish for many original OpenJPA developers to collect such detailed statistics, so that based on the access pattern of correlated fields we can prescribe which fetch groups will best serve an applications need. A nit: to determine whether a field is persistent, the patch code checks AccessCode and if the bean-method starts with "getX" etc. That check should include boolean fields which may start with "is". You should find the logic somewhere (do not remember exactly where, but around AccessCode or classes that use it), some code that determines whether a named field is persistent.
        Hide
        Pinaki Poddar added a comment -

        The documentation refers the availability of this feature or OpenJPA Tools repository.
        I think this is the first time we are referring to OpenJPA tools from our main documentation pages.

        Several useful tools in the past has been floating around. Please open a separate discussion thread on how best we could organize our tools and ensure that they reach the users.

        Show
        Pinaki Poddar added a comment - The documentation refers the availability of this feature or OpenJPA Tools repository. I think this is the first time we are referring to OpenJPA tools from our main documentation pages. Several useful tools in the past has been floating around. Please open a separate discussion thread on how best we could organize our tools and ensure that they reach the users.
        Hide
        Rick Curtis added a comment -

        > A nit: to ...
        Nice catch on the isXXXX.

        > Several useful tools in the past has been floating around. Please open a separate discussion thread on how best we could organize our tools and ensure that they reach the users.
        I thought we came the consensus that if the tool has a dependency on a version of the runtime, it needs to live with the runtime code?. If it has no runtime dependency, it would live in the seperate openjpa-tools repository?

        Show
        Rick Curtis added a comment - > A nit: to ... Nice catch on the isXXXX. > Several useful tools in the past has been floating around. Please open a separate discussion thread on how best we could organize our tools and ensure that they reach the users. I thought we came the consensus that if the tool has a dependency on a version of the runtime, it needs to live with the runtime code?. If it has no runtime dependency, it would live in the seperate openjpa-tools repository?
        Hide
        Rick Curtis added a comment -

        We should also exclude version and primary key fields as I could see where those fields aren't used directly by application code, but they are still important to the runtime.

        Show
        Rick Curtis added a comment - We should also exclude version and primary key fields as I could see where those fields aren't used directly by application code, but they are still important to the runtime.
        Hide
        Helen Xu added a comment - - edited

        Thanks for your comments.

        I'll make the code change to exclude the PK fields. Version fields are excluded.

        For the boolean field, the enhancer also generate the getXXXX method. it is covered in the test case.

        Show
        Helen Xu added a comment - - edited Thanks for your comments. I'll make the code change to exclude the PK fields. Version fields are excluded. For the boolean field, the enhancer also generate the getXXXX method. it is covered in the test case.
        Hide
        Pinaki Poddar added a comment -

        Rick,
        > we came the consensus that if the tool has a dependency on a version of the runtime, ....
        I am inviting a discussion on tools to get a better focus/attention of other members on such activity. The tools are "unsupported" in some way – meaning the processes (distribution, documentation, release) around them arr not as matured as that of OpenJPA.

        Show
        Pinaki Poddar added a comment - Rick, > we came the consensus that if the tool has a dependency on a version of the runtime, .... I am inviting a discussion on tools to get a better focus/attention of other members on such activity. The tools are "unsupported" in some way – meaning the processes (distribution, documentation, release) around them arr not as matured as that of OpenJPA.
        Hide
        Rick Curtis added a comment -

        After thinking more about this last night I'm thinking we could also exclude fields that are currently marked lazy.... telling the user that they didn't access a field marked as LAZY seems like noise. Thoughts?

        > The tools are "unsupported" in some way – meaning the processes (distribution, documentation, release) around them arr not as matured as that of OpenJPA.
        Agreed.

        Show
        Rick Curtis added a comment - After thinking more about this last night I'm thinking we could also exclude fields that are currently marked lazy.... telling the user that they didn't access a field marked as LAZY seems like noise. Thoughts? > The tools are "unsupported" in some way – meaning the processes (distribution, documentation, release) around them arr not as matured as that of OpenJPA. Agreed.
        Hide
        Rick Curtis added a comment -

        I committed revision 1356860 to trunk.

        This first iteration of this tool is limited in usage. Currently it is not configurable. Once it is on the classpath, it starts running. When you take it off the classpath, it stops running. It is also depended upon runtime (ie: javaagent or container enhancement) enhancement.

        I modified a number of things from the original patch to make it more usable... When the code first gets loaded up you'll see this message[1] so a user is notified when it is running. Also, when the statistics are dumped you'll see a message along these lines[2].

        I'll also note that I modified the pom to spit out another jar file that allows this tool to work in a WebSphere environment.

        [1] 3094 fetchStatisticPU INFO [main] openjpa.Runtime - The FetchStatisticsAuxEnhancer has been loaded and is tracking persistent field usage.
        [2] 4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics from Entities [org.apache.openjpa.enhance.stats.AEntity,..., org.apache.openjpa.enhance.stats.EEntity]. The following fields are FetchType.EAGER and were never fetched [ total 10 ] :
        org.apache.openjpa.enhance.stats.AEntity.checked
        org.apache.openjpa.enhance.stats.AEntity.desc
        org.apache.openjpa.enhance.stats.AEntity.extraInfo
        org.apache.openjpa.enhance.stats.AEntity.name
        ...

        Show
        Rick Curtis added a comment - I committed revision 1356860 to trunk. This first iteration of this tool is limited in usage. Currently it is not configurable. Once it is on the classpath, it starts running. When you take it off the classpath, it stops running. It is also depended upon runtime (ie: javaagent or container enhancement) enhancement. I modified a number of things from the original patch to make it more usable... When the code first gets loaded up you'll see this message [1] so a user is notified when it is running. Also, when the statistics are dumped you'll see a message along these lines [2] . I'll also note that I modified the pom to spit out another jar file that allows this tool to work in a WebSphere environment. [1] 3094 fetchStatisticPU INFO [main] openjpa.Runtime - The FetchStatisticsAuxEnhancer has been loaded and is tracking persistent field usage. [2] 4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics from Entities [org.apache.openjpa.enhance.stats.AEntity,..., org.apache.openjpa.enhance.stats.EEntity] . The following fields are FetchType.EAGER and were never fetched [ total 10 ] : org.apache.openjpa.enhance.stats.AEntity.checked org.apache.openjpa.enhance.stats.AEntity.desc org.apache.openjpa.enhance.stats.AEntity.extraInfo org.apache.openjpa.enhance.stats.AEntity.name ...
        Hide
        Helen Xu added a comment - - edited

        removed duplicate info from README doc and added the fields filtering condition. patch attached.

        Show
        Helen Xu added a comment - - edited removed duplicate info from README doc and added the fields filtering condition. patch attached.
        Hide
        Helen Xu added a comment -

        if user has a lot of entities, the following message could be very long.

        4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics from Entities [org.apache.openjpa.enhance.stats.AEntity,..., org.apache.openjpa.enhance.stats.EEntity]. The following fields are FetchType.EAGER and were never fetched [ total 10 ] :

        how about this :

        4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics. The following fields are FetchType.EAGER and were never fetched [ total 10 ]
        org.apache.openjpa.enhance.stats.AEntity [4]
        checked
        desc
        extraInfo
        name
        org.apache.openjpa.enhance.stats.BEntity [0]
        org.apache.openjpa.enhance.stats.CEntity.[2]
        name
        desc
        org.apache.openjpa.enhance.stats.DEntity [0]

        Show
        Helen Xu added a comment - if user has a lot of entities, the following message could be very long. 4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics from Entities [org.apache.openjpa.enhance.stats.AEntity,..., org.apache.openjpa.enhance.stats.EEntity] . The following fields are FetchType.EAGER and were never fetched [ total 10 ] : how about this : 4226 fetchStatisticPU INFO [Thread-5] openjpa.Runtime - Successfully collected fetch statistics. The following fields are FetchType.EAGER and were never fetched [ total 10 ] org.apache.openjpa.enhance.stats.AEntity [4] checked desc extraInfo name org.apache.openjpa.enhance.stats.BEntity [0] org.apache.openjpa.enhance.stats.CEntity. [2] name desc org.apache.openjpa.enhance.stats.DEntity [0]

          People

          • Assignee:
            Rick Curtis
            Reporter:
            Helen Xu
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development