Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6
    • Fix Version/s: 0.6
    • Labels:
      None

      Description

      Note: this ticket has become a ticket about improving the performance of the platform

      In order to track performance improvements, we need some reproducible performance benchmarks. Here are some ideas of what we'd need:

      • use PEs that do nothing but create a new message and forward. Allows us to focus on the overhead of the platform
      • what is the maximum throughput without dropping messages, in a given host (in a setup with 1 adapter node and 1 or 2 app nodes)
      • what is the latency for end to end processing (avg, median, etc...)
      • using a very simple app, with only 1 PE prototype
      • varying the number of keys
      • using a slightly more complex app (at least 2 communicating prototypes), in order to take into account inter-PE communications and related optimizations
      • start measurements after a warmup phase

      Some tests could be part of the test suite (by specifying a given option for those performance-related tests). That would allow some tracking of the performance.

      We could also add a simple injection mechanism that would work out of the box with the example bundled with new S4 apps (through "s4 newApp" command).

        Issue Links

          Activity

          Matthieu Morel created issue -
          Matthieu Morel made changes -
          Field Original Value New Value
          Description In order to track performance improvements, we need some reproducible performance benchmarks. Here are some ideas of what we'd need:

          - what is the maximum throughput without dropping messages, in a given host (in a setup with 1 adapter node and 1 or 2 app nodes)
          - using a very simple app, with only 1 PE prototype
          - varying the number of keys
          - using a slightly more complex app (at least 2 communicating prototypes), in order to take into account inter-PE communications and related optimizations

          Some tests could be part of the test suite (by specifying a given option for those performance-related tests). That would allow some tracking of the performance.

          We could also add a simple injection mechanism that would work out of the box with the example bundled with new S4 apps (through "s4 newApp" command).
          In order to track performance improvements, we need some reproducible performance benchmarks. Here are some ideas of what we'd need:

          - use PEs that do nothing but create a new message and forward. Allows us to focus on the overhead of the platform
          - what is the maximum throughput without dropping messages, in a given host (in a setup with 1 adapter node and 1 or 2 app nodes)
          - what is the latency for end to end processing (avg, median, etc...)
          - using a very simple app, with only 1 PE prototype
          - varying the number of keys
          - using a slightly more complex app (at least 2 communicating prototypes), in order to take into account inter-PE communications and related optimizations
          - start measurements after a warmup phase

          Some tests could be part of the test suite (by specifying a given option for those performance-related tests). That would allow some tracking of the performance.

          We could also add a simple injection mechanism that would work out of the box with the example bundled with new S4 apps (through "s4 newApp" command).
          Matthieu Morel made changes -
          Link This issue contains S4-86 [ S4-86 ]
          Matthieu Morel made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Matthieu Morel [ mmorel ]
          Matthieu Morel made changes -
          Link This issue contains S4-94 [ S4-94 ]
          Matthieu Morel made changes -
          Link This issue relates to S4-107 [ S4-107 ]
          Matthieu Morel made changes -
          Link This issue contains S4-8 [ S4-8 ]
          Matthieu Morel made changes -
          Summary Add reproducible performance benchmarks Performance improvements
          Matthieu Morel made changes -
          Description In order to track performance improvements, we need some reproducible performance benchmarks. Here are some ideas of what we'd need:

          - use PEs that do nothing but create a new message and forward. Allows us to focus on the overhead of the platform
          - what is the maximum throughput without dropping messages, in a given host (in a setup with 1 adapter node and 1 or 2 app nodes)
          - what is the latency for end to end processing (avg, median, etc...)
          - using a very simple app, with only 1 PE prototype
          - varying the number of keys
          - using a slightly more complex app (at least 2 communicating prototypes), in order to take into account inter-PE communications and related optimizations
          - start measurements after a warmup phase

          Some tests could be part of the test suite (by specifying a given option for those performance-related tests). That would allow some tracking of the performance.

          We could also add a simple injection mechanism that would work out of the box with the example bundled with new S4 apps (through "s4 newApp" command).
          bq. *Note: this ticket has become a ticket about improving the performance of the platform*

          In order to track performance improvements, we need some reproducible performance benchmarks. Here are some ideas of what we'd need:

          - use PEs that do nothing but create a new message and forward. Allows us to focus on the overhead of the platform
          - what is the maximum throughput without dropping messages, in a given host (in a setup with 1 adapter node and 1 or 2 app nodes)
          - what is the latency for end to end processing (avg, median, etc...)
          - using a very simple app, with only 1 PE prototype
          - varying the number of keys
          - using a slightly more complex app (at least 2 communicating prototypes), in order to take into account inter-PE communications and related optimizations
          - start measurements after a warmup phase

          Some tests could be part of the test suite (by specifying a given option for those performance-related tests). That would allow some tracking of the performance.

          We could also add a simple injection mechanism that would work out of the box with the example bundled with new S4 apps (through "s4 newApp" command).
          Matthieu Morel made changes -
          Issue Type Test [ 6 ] Improvement [ 4 ]
          Matthieu Morel made changes -
          Link This issue contains S4-106 [ S4-106 ]
          Matthieu Morel made changes -
          Link This issue contains S4-99 [ S4-99 ]
          Matthieu Morel made changes -
          Fix Version/s 0.6 [ 12321702 ]
          Matthieu Morel made changes -
          Link This issue contains S4-48 [ S4-48 ]
          Matthieu Morel made changes -
          Link This issue contains S4-62 [ S4-62 ]
          Matthieu Morel made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Karthik Kambatla (Inactive) made changes -
          Link This issue incorporates S4-109 [ S4-109 ]

            People

            • Assignee:
              Matthieu Morel
              Reporter:
              Matthieu Morel
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development