Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      I'd like to propose that we provide a command line utility similar to play in Play! Framework. The idea is to implement various common function using a single utility program. Examples:

      > s4 build // builds an app
      > s4 load // loads an app to the cluster.
      > s4 unload // stops and unloads an app from the cluster
      > s4 stats // provide stats about a running cluster
      

      We can start by modifying the Play script which is under Apache license.

        Activity

        Hide
        Leo Neumeyer added a comment -

        Put a skeleton copied from the Play! Framework here:
        https://github.com/leoneu/s4-command

        Here is the list of Play commands. I put a star to the ones that are applicable to S4. (We don't necessarily need to support all of them but it matches pretty well with what we need.)

        ~ Core commands:
        ~ ~~~~~~~~~~~~~~
        ~ antify Create a build.xml file for this project *
        ~ auto-test Automatically run all application tests *
        ~ build-module Build and package a module
        ~ check Check for a release newer than the current one *
        ~ classpath Display the computed classpath *
        ~ clean Delete temporary files (including the bytecode cache) *
        ~ dependencies Resolve and retrieve project dependencies *
        ~ eclipsify Create all Eclipse configuration files *
        ~ evolutions Run the evolution check
        ~ evolutions:applyAutomatically apply pending evolutions
        ~ evolutions:markAppliedMark pending evolutions as manually applied
        ~ evolutions:resolveResolve partially applied evolution
        ~ help Display help on a specific command *
        ~ id Define the framework ID *
        ~ idealize Create all IntelliJ Idea configuration files *
        ~ install Install a module
        ~ javadoc Generate your application Javadoc *
        ~ list-modules List modules available from the central modules repository
        ~ modules Display the computed modules list
        ~ netbeansify Create all NetBeans configuration files *
        ~ new Create a new application *
        ~ new-module Create a module
        ~ out Follow logs/system.out file *
        ~ pid Show the PID of the running application *
        ~ precompile Precompile all Java sources and templates to speed up application start-up
        ~ restart Restart the running application *
        ~ run Run the application in the current shell *
        ~ secret Generate a new secret key *
        ~ start Start the application in the background *
        ~ status Display the running application's status *
        ~ stop Stop the running application *
        ~ test Run the application in test mode in the current shell *
        ~ version Print the framework version *
        ~ war Export the application as a standalone WAR archive * (as an s4r)

        Show
        Leo Neumeyer added a comment - Put a skeleton copied from the Play! Framework here: https://github.com/leoneu/s4-command Here is the list of Play commands. I put a star to the ones that are applicable to S4. (We don't necessarily need to support all of them but it matches pretty well with what we need.) ~ Core commands: ~ ~~~~~~~~~~~~~~ ~ antify Create a build.xml file for this project * ~ auto-test Automatically run all application tests * ~ build-module Build and package a module ~ check Check for a release newer than the current one * ~ classpath Display the computed classpath * ~ clean Delete temporary files (including the bytecode cache) * ~ dependencies Resolve and retrieve project dependencies * ~ eclipsify Create all Eclipse configuration files * ~ evolutions Run the evolution check ~ evolutions:applyAutomatically apply pending evolutions ~ evolutions:markAppliedMark pending evolutions as manually applied ~ evolutions:resolveResolve partially applied evolution ~ help Display help on a specific command * ~ id Define the framework ID * ~ idealize Create all IntelliJ Idea configuration files * ~ install Install a module ~ javadoc Generate your application Javadoc * ~ list-modules List modules available from the central modules repository ~ modules Display the computed modules list ~ netbeansify Create all NetBeans configuration files * ~ new Create a new application * ~ new-module Create a module ~ out Follow logs/system.out file * ~ pid Show the PID of the running application * ~ precompile Precompile all Java sources and templates to speed up application start-up ~ restart Restart the running application * ~ run Run the application in the current shell * ~ secret Generate a new secret key * ~ start Start the application in the background * ~ status Display the running application's status * ~ stop Stop the running application * ~ test Run the application in test mode in the current shell * ~ version Print the framework version * ~ war Export the application as a standalone WAR archive * (as an s4r)
        Hide
        Matthieu Morel added a comment -

        Nice, plenty of good things there!

        That will surely facilitate development of S4 applications.

        And my understanding is that more advanced admin facilities would be a separate set of scripts, right?

        So a typical scenario would be:

        1. create an application:
          s4 new <app-dir>

          This creates a project skeleton in the specified directory, along with a gradle build script for that project.

        2. optionally prepare for editing in an IDE:
          s4 eclipsify|idealize <app-dir> 
        3. the developer adds application code in this project (and possibly adds dependencies in the gradle script if needed)
        4. build and package
           s4 s4r <app-dir> 

          This creates an S4R archive

        5. start a cluster
           s4 start 

          (we might add parameters such as the number of nodes).
          This starts S4 nodes locally, a Zookeeper instance, and create a logical cluster with the running S4 nodes (for instance by default a 2 nodes cluster)

        6. deploy to the cluster
           s4 deploy <path-to-s4r> 

          or

           s4 deploy <app-dir> 

          This deploys the application to the running local S4 cluster

        Show
        Matthieu Morel added a comment - Nice, plenty of good things there! That will surely facilitate development of S4 applications. And my understanding is that more advanced admin facilities would be a separate set of scripts, right? So a typical scenario would be: create an application: s4 new <app-dir> This creates a project skeleton in the specified directory, along with a gradle build script for that project. optionally prepare for editing in an IDE: s4 eclipsify|idealize <app-dir> the developer adds application code in this project (and possibly adds dependencies in the gradle script if needed) build and package s4 s4r <app-dir> This creates an S4R archive start a cluster s4 start (we might add parameters such as the number of nodes). This starts S4 nodes locally, a Zookeeper instance, and create a logical cluster with the running S4 nodes (for instance by default a 2 nodes cluster) deploy to the cluster s4 deploy <path-to-s4r> or s4 deploy <app-dir> This deploys the application to the running local S4 cluster
        Hide
        Matthieu Morel added a comment -

        I have implemented a simple tool for running the twitter example in S4-22-rebased branch
        See: https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git;a=blob;f=s4;h=a958cc2149cd54bb5d55bdda6522c013827c57f8;hb=8931c63c8957fcbd723cce6664d425d0ce3c6cfe

        I added a set of classes in the s4-tools subproject for handling the different tasks. They also include parameter parsing and validation through jcommander.

        You can generate a proper script for running those tools by calling "gradle s4-tools:installApp"

        "s4" then simply calls this tool script with some parameters passed by the user.

        Show
        Matthieu Morel added a comment - I have implemented a simple tool for running the twitter example in S4-22 -rebased branch See: https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git;a=blob;f=s4;h=a958cc2149cd54bb5d55bdda6522c013827c57f8;hb=8931c63c8957fcbd723cce6664d425d0ce3c6cfe I added a set of classes in the s4-tools subproject for handling the different tasks. They also include parameter parsing and validation through jcommander. You can generate a proper script for running those tools by calling "gradle s4-tools:installApp" "s4" then simply calls this tool script with some parameters passed by the user.
        Hide
        Matthieu Morel added a comment -

        This is implemented in S4-22, and used through the walkthrough : https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough

        Show
        Matthieu Morel added a comment - This is implemented in S4-22 , and used through the walkthrough : https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough
        Hide
        Matthieu Morel added a comment -

        We still need to add:

        • s4 eclipsify|idealize <app-dir>

        It would also be nice to have something like:

        • s4 status

        that would return the runtime information about the system: subclusters, tasks/nodes/apps per subclusters, published streams

        • s4 undeploy <app-name>

        (undeploy has implications about classloading, and we might change a few things there, see S4-4 ).

        Show
        Matthieu Morel added a comment - We still need to add: s4 eclipsify|idealize <app-dir> It would also be nice to have something like: s4 status that would return the runtime information about the system: subclusters, tasks/nodes/apps per subclusters, published streams s4 undeploy <app-name> (undeploy has implications about classloading, and we might change a few things there, see S4-4 ).
        Hide
        Matthieu Morel added a comment -

        we now have the following commands:

        • deploy (publishes app in Zookeeper config)
        • node (starts an S4 node)
        • zkServer (starts a Zookeeper instance for easy testing)
        • newCluster (configures a S4 subcluster)
        • adapter (starts an adapter node for easy testing)
        • s4r (packages an S4 application)
        • status (provides information about the status of clusters, apps and streams)

        We now have a decent toolset that provides all required functionality. I am therefore marking this ticket as resolved.

        Other commands to add will be addressed in new tickets.

        Show
        Matthieu Morel added a comment - we now have the following commands: deploy (publishes app in Zookeeper config) node (starts an S4 node) zkServer (starts a Zookeeper instance for easy testing) newCluster (configures a S4 subcluster) adapter (starts an adapter node for easy testing) s4r (packages an S4 application) status (provides information about the status of clusters, apps and streams) We now have a decent toolset that provides all required functionality. I am therefore marking this ticket as resolved. Other commands to add will be addressed in new tickets.

          People

          • Assignee:
            Unassigned
            Reporter:
            Leo Neumeyer
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development