Apache S4
  1. Apache S4
  2. S4-11

add checkpointing mechanism to s4-piper

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      There is already such a mechanism in S4.

      I'll port it to S4-piper, once the API is more stable, especially regarding triggers.

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          274d 5h 23m 1 Matthieu Morel 18/Jul/12 17:17
          Patch Available Patch Available Resolved Resolved
          1d 21h 54m 1 Matthieu Morel 20/Jul/12 15:12
          Matthieu Morel made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Matthieu Morel added a comment -

          merged into piper branch commit 54c5fa232b6bead60f2c2f27286cf2c81f18d4ea

          Show
          Matthieu Morel added a comment - merged into piper branch commit 54c5fa232b6bead60f2c2f27286cf2c81f18d4ea
          Hide
          Daniel Gómez Ferro added a comment -

          This is a great feature and patch looks good, +1!

          Show
          Daniel Gómez Ferro added a comment - This is a great feature and patch looks good, +1!
          Hide
          Matthieu Morel added a comment -

          Just updated the patch with a clarification of the API for evaluating the "dirty" state of the API, following a suggestion from Daniel. See commit 726580ba162ffaa4107ee4c89c16b139cfa520d8

          Show
          Matthieu Morel added a comment - Just updated the patch with a clarification of the API for evaluating the "dirty" state of the API, following a suggestion from Daniel. See commit 726580ba162ffaa4107ee4c89c16b139cfa520d8
          Matthieu Morel made changes -
          Link This issue contains S4-44 [ S4-44 ]
          Matthieu Morel made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.5 [ 12318653 ]
          Hide
          Matthieu Morel added a comment -

          Patch available in branch S4-11.

          Provides a checkpointing mechanism with the following characteristics:

          • based either on event count or on time intervals
          • configurable per PE prototype
          • parallel with event processing flow
          • default backend implementation relying on a file system is provided
          • backend configuration through Guice modules
          • configurable timeout on fetches (S4-44)

          Patch also updates the twitter trending example by adding checkpointing capabilities.

          Note that checkpointing, to be enabled, requires both:

          • the application to specify the checkpointing configuration per prototype
          • the S4 node configuration to enable checkpointing (configured through a module, a default one with a file system backend is available in s4-core)
          Show
          Matthieu Morel added a comment - Patch available in branch S4-11 . Provides a checkpointing mechanism with the following characteristics: based either on event count or on time intervals configurable per PE prototype parallel with event processing flow default backend implementation relying on a file system is provided backend configuration through Guice modules configurable timeout on fetches ( S4-44 ) Patch also updates the twitter trending example by adding checkpointing capabilities. Note that checkpointing, to be enabled, requires both: the application to specify the checkpointing configuration per prototype the S4 node configuration to enable checkpointing (configured through a module, a default one with a file system backend is available in s4-core)
          Tony Stevenson made changes -
          Workflow jira [ 12638185 ] no-reopen-closed, patch-avail [ 12711313 ]
          Matthieu Morel made changes -
          Field Original Value New Value
          Fix Version/s 0.5 [ 12318653 ]
          Matthieu Morel created issue -

            People

            • Assignee:
              Matthieu Morel
              Reporter:
              Matthieu Morel
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development