Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Testing
    • Labels:
      None

      Description

      As described/discussed in [0] it would be nice to have a performance test tool in Sling .
      This can be useful in different situations (e.g. micro benchmarks of a feature and so on).

      Patch to follow

      [0] http://sling.markmail.org/message/bz44im7aqeae4r57

      1. SLING-2311-patch.txt
        32 kB
        Antonio Sanso
      2. ResolveWithManyVanityPath.png
        24 kB
        Antonio Sanso
      3. ResolveWithManyAliasTest.java
        11 kB
        Antonio Sanso
      4. plot.sh
        2 kB
        Antonio Sanso
      5. plot.sh
        2 kB
        Antonio Sanso

        Activity

        Hide
        Justin Edelson added a comment -

        finally added plot.sh in r1392349

        current report here: http://people.apache.org/~justin/sling-perf-reports/2012-Oct-01/report.html

        Show
        Justin Edelson added a comment - finally added plot.sh in r1392349 current report here: http://people.apache.org/~justin/sling-perf-reports/2012-Oct-01/report.html
        Hide
        Antonio Sanso added a comment -

        @justing you are right... I'll give a look into it...

        Show
        Antonio Sanso added a comment - @justing you are right... I'll give a look into it...
        Hide
        Justin Edelson added a comment -

        @Antonio - do you know how to make the generated graphs wider? The X axis labels are overlapping a bit and it'd be nice to fix them.

        Thanks for your help with this!

        Show
        Justin Edelson added a comment - @Antonio - do you know how to make the generated graphs wider? The X axis labels are overlapping a bit and it'd be nice to fix them. Thanks for your help with this!
        Hide
        Antonio Sanso added a comment -

        I temporary put the report here

        http://people.apache.org/~asanso/sling/report-2012-08-31/report.html

        until we find a better location...

        Show
        Antonio Sanso added a comment - I temporary put the report here http://people.apache.org/~asanso/sling/report-2012-08-31/report.html until we find a better location...
        Hide
        Antonio Sanso added a comment -

        @justin
        apologies I did update an obsolete version. I had the same issue so I took a shortcut I skipped the conversion and referenced directly svg in the html.

        uploaded new version

        Show
        Antonio Sanso added a comment - @justin apologies I did update an obsolete version. I had the same issue so I took a shortcut I skipped the conversion and referenced directly svg in the html. uploaded new version
        Hide
        Justin Edelson added a comment -

        @Antonio - I can't get this plot.sh script to work correctly. How did you install Gnuplot? Any other requirements?

        I can get the svg files, but the png conversion doesn't work. Any ideas?

        Show
        Justin Edelson added a comment - @Antonio - I can't get this plot.sh script to work correctly. How did you install Gnuplot? Any other requirements? I can get the svg files, but the png conversion doesn't work. Any ideas?
        Hide
        Antonio Sanso added a comment -

        You are right JcrResourceListener was a left over for something else (namely adding vanityPath on the fly) so it can be safely be removed (for now). Thanks

        Show
        Antonio Sanso added a comment - You are right JcrResourceListener was a left over for something else (namely adding vanityPath on the fly) so it can be safely be removed (for now). Thanks
        Hide
        Antonio Sanso added a comment -

        @justin thanks for committing and improving this stuff. Attaching the plot.sh file (again credit to Jackrabbit team) should we want to aim for something like
        http://people.apache.org/~jukka/jackrabbit/report-2011-08-25/report.html

        Show
        Antonio Sanso added a comment - @justin thanks for committing and improving this stuff. Attaching the plot.sh file (again credit to Jackrabbit team) should we want to aim for something like http://people.apache.org/~jukka/jackrabbit/report-2011-08-25/report.html
        Hide
        Justin Edelson added a comment -

        changes made for SLING-2579 required some slight tweaking. done in r1377828

        Show
        Justin Edelson added a comment - changes made for SLING-2579 required some slight tweaking. done in r1377828
        Hide
        Justin Edelson added a comment -

        I've added this in r1377819. Thanks for the contribution.

        I had to make a number of modifications. Chiefly is that since these tests rely on internal methods which change from release to release, the logic for constructing a ResourceResolver is encapsulated into a helper class (called TestHelper - don't love the name, but we can change it).

        Added more scenarios for the tests included based on node size, e.g. 1000 vanity paths, 5000 vanity paths, etc.

        One thing that I couldn't figure out is why a JcrResourceListener was being created, so I removed that bit. Antonio - please let me know if this needs to be there.

        Tests can be run for JCR Resource Resolver 2.0.10, 2.1.0, and 2.1.1-SNAPSHOT (which I'm calling 2.2.0 for the sake of argument).

        Show
        Justin Edelson added a comment - I've added this in r1377819. Thanks for the contribution. I had to make a number of modifications. Chiefly is that since these tests rely on internal methods which change from release to release, the logic for constructing a ResourceResolver is encapsulated into a helper class (called TestHelper - don't love the name, but we can change it). Added more scenarios for the tests included based on node size, e.g. 1000 vanity paths, 5000 vanity paths, etc. One thing that I couldn't figure out is why a JcrResourceListener was being created, so I removed that bit. Antonio - please let me know if this needs to be there. Tests can be run for JCR Resource Resolver 2.0.10, 2.1.0, and 2.1.1-SNAPSHOT (which I'm calling 2.2.0 for the sake of argument).
        Hide
        Antonio Sanso added a comment -

        IMHO junit-perf is cool, point is that sometimes is not enough to decorate a unit test since you need
        to create a specific 'scenario' e.g.for SLING-2255 the prerequisite was to have an high number of
        vanity paths.

        About a new module as said this is a 'porting' of something already existing in jackrabbit and I
        have tried to maintain the original idea just adapting a bit the framework to be more
        bundle version friendly rather than release based. An example of a potential outcome is in [0].

        [0] https://issues.apache.org/jira/secure/attachment/12520603/ResolveWithManyVanityPath.png

        Show
        Antonio Sanso added a comment - IMHO junit-perf is cool, point is that sometimes is not enough to decorate a unit test since you need to create a specific 'scenario' e.g.for SLING-2255 the prerequisite was to have an high number of vanity paths. About a new module as said this is a 'porting' of something already existing in jackrabbit and I have tried to maintain the original idea just adapting a bit the framework to be more bundle version friendly rather than release based. An example of a potential outcome is in [0] . [0] https://issues.apache.org/jira/secure/attachment/12520603/ResolveWithManyVanityPath.png
        Hide
        Antonio Sanso added a comment -

        attaching possible outcome of this performance framework

        Show
        Antonio Sanso added a comment - attaching possible outcome of this performance framework
        Hide
        Carsten Ziegeler added a comment -

        Great stuff - I think we should definitely have something like this, not sure whether we should start a new module or add it to commons testing or could just test
        junit-perf for this?

        Show
        Carsten Ziegeler added a comment - Great stuff - I think we should definitely have something like this, not sure whether we should start a new module or add it to commons testing or could just test junit-perf for this?
        Hide
        Antonio Sanso added a comment - - edited

        Adding new test case covering the scenario discussed in [0] sling:alias support.
        It tests node having 1000 children and resource not found.
        Here the outcome of the test with sling:alias support enabled (time in ms)

        min 10% 50% 90% max
        jcr.resource-2.0.11 51 57 76 263 546

        and with sling:alias support disabled

        min 10% 50% 90% max
        jcr.resource-2.0.11 0 0 0 1 48

        [0] http://sling.markmail.org/message/mlbw237hchhcyv6s

        Show
        Antonio Sanso added a comment - - edited Adding new test case covering the scenario discussed in [0] sling:alias support. It tests node having 1000 children and resource not found. Here the outcome of the test with sling:alias support enabled (time in ms) min 10% 50% 90% max jcr.resource-2.0.11 51 57 76 263 546 and with sling:alias support disabled min 10% 50% 90% max jcr.resource-2.0.11 0 0 0 1 48 [0] http://sling.markmail.org/message/mlbw237hchhcyv6s
        Hide
        Antonio Sanso added a comment -

        I have attached a prototype of the tool that I have used to produce [0].
        There are still a lot of things to tweak (pom files, scripts to produce the graph, etc) but it gives an idea of the approach. It is basically a 'porting' of what already exists in Jackrabbit so all credits go to the Jackrabbit project.

        [0] https://issues.apache.org/jira/secure/attachment/12500677/performance.pdf

        Show
        Antonio Sanso added a comment - I have attached a prototype of the tool that I have used to produce [0] . There are still a lot of things to tweak (pom files, scripts to produce the graph, etc) but it gives an idea of the approach. It is basically a 'porting' of what already exists in Jackrabbit so all credits go to the Jackrabbit project. [0] https://issues.apache.org/jira/secure/attachment/12500677/performance.pdf

          People

          • Assignee:
            Justin Edelson
            Reporter:
            Antonio Sanso
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development