Details

    • Type: Improvement Improvement
    • Status: Reopened
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Extensions
    • Labels:
      None

      Description

      In some cases (like the webloader example), long-running processes need to be started, monitored and stopped.

      The webloader implements this in a naive way, it might be useful to have a more generic facility for this: a service that would:

      1) Start a script or servlet, probably passing it a fake request object that gives access to parameters and output but is not a real HTTP request

      2) Display a status page where currently running jobs can be monitored, and stopped if desired

      3) Collect the output of such jobs in the repository and give access to it via a simple monitoring interface

      The output of long-running jobs could be structured using html conventions (like <div class="status">running step 3 of 12</div>) to create overview displays of all currently running jobs.

      This is just an idea for now, I'm not going to work on this right away, but it's probably good to keep in our wishlist.

        Issue Links

          Activity

          Hide
          Carsten Ziegeler added a comment -

          I think it's time to look at the api of this feature, I see a lot of classes in the org.apache.sling.bgservlets which really shouldn't be public:

          • BackgroundHttpServletReq/Res are implementation details
          • RuntimeState is not used. Remove?
          • Predicate is not really used. Remove?
          • JobData should not be public
          • JobStorage should not be public

          Then we have the mgmt api: ExecutionEngine and JobConsole - I think we just need one of them. In addition:

          • ExecutionEngine shouldn't allow to start a runnable, it should just provide status (maybe rename)
          • ExecutionEngine#getMatchingJobStatus(Predicate) can be simplyfied to return all status objects
          • ExecutionEngine#getJobStatus(path) - how do I now the path? Maybe remove this
          • obConsole shouldn't use a jcr session in the api; ResourceResolver maybe?
          Show
          Carsten Ziegeler added a comment - I think it's time to look at the api of this feature, I see a lot of classes in the org.apache.sling.bgservlets which really shouldn't be public: BackgroundHttpServletReq/Res are implementation details RuntimeState is not used. Remove? Predicate is not really used. Remove? JobData should not be public JobStorage should not be public Then we have the mgmt api: ExecutionEngine and JobConsole - I think we just need one of them. In addition: ExecutionEngine shouldn't allow to start a runnable, it should just provide status (maybe rename) ExecutionEngine#getMatchingJobStatus(Predicate) can be simplyfied to return all status objects ExecutionEngine#getJobStatus(path) - how do I now the path? Maybe remove this obConsole shouldn't use a jcr session in the api; ResourceResolver maybe?
          Hide
          Carsten Ziegeler added a comment -

          Changed the filter order to -1000000000 to be able to run a filter in front in revision 1146706

          Show
          Carsten Ziegeler added a comment - Changed the filter order to -1000000000 to be able to run a filter in front in revision 1146706
          Hide
          Felix Meschberger added a comment -

          Adapted to use the new SlingRequestProcessor service from SLING-1603 in Rev. 993282

          Show
          Felix Meschberger added a comment - Adapted to use the new SlingRequestProcessor service from SLING-1603 in Rev. 993282
          Hide
          Bertrand Delacretaz added a comment -

          With revision #982264, background servlets get a RuntimeState interface as a request attribute, that they can use to report runtime state to the engine.

          The BackgroundTestServlet [1] example uses that feature.

          [1] http://svn.apache.org/repos/asf/sling/trunk/contrib/extensions/bgservlets/src/main/java/org/apache/sling/bgservlets/impl/servlets/BackgroundTestServlet.java

          Show
          Bertrand Delacretaz added a comment - With revision #982264, background servlets get a RuntimeState interface as a request attribute, that they can use to report runtime state to the engine. The BackgroundTestServlet [1] example uses that feature. [1] http://svn.apache.org/repos/asf/sling/trunk/contrib/extensions/bgservlets/src/main/java/org/apache/sling/bgservlets/impl/servlets/BackgroundTestServlet.java
          Hide
          Bertrand Delacretaz added a comment -

          Revision 979727 provides the following features:

          Any servlet executed by Sling can be started in the background by adding the sling:bg=true parameter to the request. The servlet then gets a "fake" request object, some servlets might not like that.

          Background servlets are executed by a simple (and limited) thread pool.

          Nodes corresponding to jobs are stored under /var/bg/jobs with a sling:bgJobData mixin and the sling/bg/job resource type. The output of a background servlet is stored in a node named "stream", with the sling/bg/stream resource type, under that node.

          Default servlets are provided to render the status and stream nodes, when starting a servlet in the background Sling redirects to the status node, using the same extension as the request.

          How to test:
          Use the engine bundle from the SLING-1603 branch [1].

          Install the contrib/extensions/bgservlets bundle.

          Start the test servlet in the background using either:
          http://localhost:8080/system/bgservlets/test.html?sling:bg=true
          http://localhost:8080/system/bgservlets/test.txt?sling:bg=true
          http://localhost:8080/system/bgservlets/test.json?sling:bg=true

          Sling returns the job status page, with a link to the job stream. Refreshing the job stream URL returns the full stream as it progresses, saved on every flush() of the running servlet's OutputStream.

          Jobs are listed at /system/console/bgservlets which allows them to be suspended, resumed and stopped.

          TODO:
          Use existing thread pool and/or scheduling code from Sling.

          Access control: status nodes should be readable by their owner only (+ admin of course).

          Replace hardcoded values by configuration properties (see TODO in code).

          ExecutionEngine should sort jobs by creation date, descending.

          Console should allow for identifying and restarting jobs that were interrupted by a shutdown.

          [1] http://svn.apache.org/repos/asf/sling/branches/SLING-1603-engine

          Show
          Bertrand Delacretaz added a comment - Revision 979727 provides the following features: Any servlet executed by Sling can be started in the background by adding the sling:bg=true parameter to the request. The servlet then gets a "fake" request object, some servlets might not like that. Background servlets are executed by a simple (and limited) thread pool. Nodes corresponding to jobs are stored under /var/bg/jobs with a sling:bgJobData mixin and the sling/bg/job resource type. The output of a background servlet is stored in a node named "stream", with the sling/bg/stream resource type, under that node. Default servlets are provided to render the status and stream nodes, when starting a servlet in the background Sling redirects to the status node, using the same extension as the request. How to test: Use the engine bundle from the SLING-1603 branch [1] . Install the contrib/extensions/bgservlets bundle. Start the test servlet in the background using either: http://localhost:8080/system/bgservlets/test.html?sling:bg=true http://localhost:8080/system/bgservlets/test.txt?sling:bg=true http://localhost:8080/system/bgservlets/test.json?sling:bg=true Sling returns the job status page, with a link to the job stream. Refreshing the job stream URL returns the full stream as it progresses, saved on every flush() of the running servlet's OutputStream. Jobs are listed at /system/console/bgservlets which allows them to be suspended, resumed and stopped. TODO: Use existing thread pool and/or scheduling code from Sling. Access control: status nodes should be readable by their owner only (+ admin of course). Replace hardcoded values by configuration properties (see TODO in code). ExecutionEngine should sort jobs by creation date, descending. Console should allow for identifying and restarting jobs that were interrupted by a shutdown. [1] http://svn.apache.org/repos/asf/sling/branches/SLING-1603-engine
          Hide
          Bertrand Delacretaz added a comment -

          Starting work on this now (it's been only two years , committed a first shot in revision 963686.

          Here's an example with the supplied test servlet, using the sling:bg parameter to run it in the background:

          $ curl -s "http://localhost:8888/system/bgservlets/test?sling:bg" | grep "in the background" | head -1

          <title>202 Running request in the background using Background request job: org.apache.sling.bgservlets.impl.ServletResponseWrapper:/var/folders/Ao/AoRgBYXQEPCEzwtY3VTbPE+++TI/Tmp/ServletResponseWrapper611074873126683939.data</title>

          $ tail f /var/folders/Ao/AoRgBYXQEPCEzwtY3VTbPE+++TI/-Tmp/ServletResponseWrapper611074873126683939.data

          Cycle 5 of 10
          Flushing output
          Cycle 6 of 10
          Cycle 7 of 10
          Flushing output
          Cycle 8 of 10
          Cycle 9 of 10
          Flushing output
          Cycle 10 of 10
          All done.

          The test servlet runs in the background and its output is written to a temporary file.

          Next steps are to implement job control, to suspend/resume/stop/restart such jobs, using a simple Felix console plugin as the UI.

          Show
          Bertrand Delacretaz added a comment - Starting work on this now (it's been only two years , committed a first shot in revision 963686. Here's an example with the supplied test servlet, using the sling:bg parameter to run it in the background: $ curl -s "http://localhost:8888/system/bgservlets/test?sling:bg" | grep "in the background" | head -1 <title>202 Running request in the background using Background request job: org.apache.sling.bgservlets.impl.ServletResponseWrapper:/var/folders/Ao/AoRgBYXQEPCEzwtY3VTbPE+++TI/ Tmp /ServletResponseWrapper611074873126683939.data</title> $ tail f /var/folders/Ao/AoRgBYXQEPCEzwtY3VTbPE+++TI/-Tmp /ServletResponseWrapper611074873126683939.data Cycle 5 of 10 Flushing output Cycle 6 of 10 Cycle 7 of 10 Flushing output Cycle 8 of 10 Cycle 9 of 10 Flushing output Cycle 10 of 10 All done. The test servlet runs in the background and its output is written to a temporary file. Next steps are to implement job control, to suspend/resume/stop/restart such jobs, using a simple Felix console plugin as the UI.
          Hide
          Bertrand Delacretaz added a comment -

          Oops - not fixed at all, wrong issue

          Show
          Bertrand Delacretaz added a comment - Oops - not fixed at all, wrong issue
          Hide
          Bertrand Delacretaz added a comment -

          Fixed in revision 669948

          Show
          Bertrand Delacretaz added a comment - Fixed in revision 669948

            People

            • Assignee:
              Bertrand Delacretaz
              Reporter:
              Bertrand Delacretaz
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development