Oozie
  1. Oozie
  2. OOZIE-1054

Create script to properly upload sharelib to HDFS

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 3.3.2
    • Component/s: scripts
    • Labels:
      None

      Description

      A common confusion-point for users is how to properly install the sharelib in HDFS. It would be really useful if we could create a script to properly upload the sharelib to HDFS so we can get rid of the confusion around this.

      Alternatively, instead of a new script, we could have the oozie-setup.sh script take an argument that does this.

      1. oozie-1054.patch
        18 kB
        Bowen Zhang
      2. oozie-1054.patch
        17 kB
        Bowen Zhang
      3. oozie-1054.patch
        18 kB
        Bowen Zhang
      4. oozie-1054.patch
        15 kB
        Bowen Zhang
      5. oozie-1054.patch
        13 kB
        Bowen Zhang
      6. oozie-1054.patch
        13 kB
        Bowen Zhang
      7. oozie-1054.patch
        13 kB
        Bowen Zhang
      8. oozie-1054.patch
        13 kB
        Bowen Zhang
      9. oozie-1054.patch
        12 kB
        Bowen Zhang
      10. oozie-1054.patch
        12 kB
        Bowen Zhang

        Activity

        Hide
        Roman Shaposhnik added a comment -

        I've been thinking about the same approach wrt. Bigtop Oozie packaging. One idea that I've entertained is to automatically 'rsync' the local copy of sharelibs with HDFS location on Oozie startup.

        Show
        Roman Shaposhnik added a comment - I've been thinking about the same approach wrt. Bigtop Oozie packaging. One idea that I've entertained is to automatically 'rsync' the local copy of sharelibs with HDFS location on Oozie startup.
        Hide
        Alejandro Abdelnur added a comment -

        I'd propose consolidating all setup operations in a single user facing script with multiple subcommands. I.e.:

        $ oozie-setup.sh [prepare-war|create-db|upgrade-db|create-sharelib|upgrade-sharelib]

        Show
        Alejandro Abdelnur added a comment - I'd propose consolidating all setup operations in a single user facing script with multiple subcommands. I.e.: $ oozie-setup.sh [prepare-war|create-db|upgrade-db|create-sharelib|upgrade-sharelib]
        Hide
        Bowen Zhang added a comment -

        I just submitted a patch that will provide an additional argument "-sharelib" to oozie-setup.sh to upload sharelib into hdfs and an "-update" flag following "-sharelib" for updating sharelib.

        Show
        Bowen Zhang added a comment - I just submitted a patch that will provide an additional argument "-sharelib" to oozie-setup.sh to upload sharelib into hdfs and an "-update" flag following "-sharelib" for updating sharelib.
        Hide
        Alejandro Abdelnur added a comment -

        Bowen Zhang, thanks for taking on this. A few comments:

        • Currently Oozie Server does not require Hadoop ('hadoop' command line tool) to be installed locally, this patch chnages that.
        • The oozie-setup.sh script, when invoked with the -sharelib option, recreates the WAR file.

        The second comment is easy to fix in the current script.

        The first one would require a new tool class to do it, then we would not need the hadoop client. The new tool, like the current DBTool would have to initialize services and use the HadoopService to get a filesystem handle, then a recursive delete() and copyFromLocal() could be used.

        Show
        Alejandro Abdelnur added a comment - Bowen Zhang, thanks for taking on this. A few comments: Currently Oozie Server does not require Hadoop ('hadoop' command line tool) to be installed locally, this patch chnages that. The oozie-setup.sh script, when invoked with the -sharelib option, recreates the WAR file. The second comment is easy to fix in the current script. The first one would require a new tool class to do it, then we would not need the hadoop client. The new tool, like the current DBTool would have to initialize services and use the HadoopService to get a filesystem handle, then a recursive delete() and copyFromLocal() could be used.
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro,
        I just want to clarify your two comments. For the first one, which current oozie "DBTool" are you referring to? I am actually quite new to this. For the second comment, are you saying when setup.sh is invoked ONLY with a "-sharelib" option, the WAR files are recreated? Because from my understanding, shouldn't you create the WAR files everytime when you run the oozie-setup.sh script? Or the command "oozie-setup.sh -sharelib sharelibFile" should merely just upload the sharelib to hdfs without doing anything else?

        Show
        Bowen Zhang added a comment - Hi Alejandro, I just want to clarify your two comments. For the first one, which current oozie "DBTool" are you referring to? I am actually quite new to this. For the second comment, are you saying when setup.sh is invoked ONLY with a "-sharelib" option, the WAR files are recreated? Because from my understanding, shouldn't you create the WAR files everytime when you run the oozie-setup.sh script? Or the command "oozie-setup.sh -sharelib sharelibFile" should merely just upload the sharelib to hdfs without doing anything else?
        Hide
        Robert Kanter added a comment -

        He's talking about the ooziedb.sh script; most of the work is actually done by a Java program (from the tools module: org.apache.oozie.tools.OozieDBCLI) that the script runs. The way you wrote the script, it relies on the hadoop CLI being available and on the PATH; instead of doing that, you can create a Java program that uses the Hadoop API to upload the sharelib and have the script run that. The advantage of doing it this way is that the user doesn't even have to have Hadoop installed on their computer, let alone having it in their PATH (because we'll have the Hadoop JARs already from Oozie). Does that make sense?

        Show
        Robert Kanter added a comment - He's talking about the ooziedb.sh script; most of the work is actually done by a Java program (from the tools module: org.apache.oozie.tools.OozieDBCLI ) that the script runs. The way you wrote the script, it relies on the hadoop CLI being available and on the PATH; instead of doing that, you can create a Java program that uses the Hadoop API to upload the sharelib and have the script run that. The advantage of doing it this way is that the user doesn't even have to have Hadoop installed on their computer, let alone having it in their PATH (because we'll have the Hadoop JARs already from Oozie). Does that make sense?
        Hide
        Bowen Zhang added a comment -

        I just submitted another patch that includes a new tool for hadoop file handler. On the command line for oozie-setup.sh, user can use "-sharelib" to specify the location of their sharelib and the fs.default.name of the file system. If a user doesn't specify fs name, it will use the default name by the config. For example, a user can use "-sharelib ~/share hdfs://localhost:9000" to upload the sharelib to hdfs.

        Show
        Bowen Zhang added a comment - I just submitted another patch that includes a new tool for hadoop file handler. On the command line for oozie-setup.sh, user can use "-sharelib" to specify the location of their sharelib and the fs.default.name of the file system. If a user doesn't specify fs name, it will use the default name by the config. For example, a user can use "-sharelib ~/share hdfs://localhost:9000" to upload the sharelib to hdfs.
        Hide
        Robert Kanter added a comment -

        Looks better:

        A few things:
        1) Make sure to generate the patch without the preceding a/ and b/ (--no-prefix for git)
        2) Some of the indenting is off in OozieFSCLI#run; make sure to use spaces, not tabs
        3) The usage information printed out by oozie-setup.sh should mention the -sharelib argument
        4) Update the documentation; make sure to mention that (by default) it should be run as the user who is going to be running the Oozie server
        5) The sharelib location is actually configurable in oozie-site.xml; OozieFSCLI should read oozie-site (see OozieDBCLI for how its done there via Services) and use the value of the oozie.service.WorkflowAppService.system.libpath property for the destination path
        6) Right now, when you run the -sharelib argument, it will prepare the war file, even if the user already did that. I think it would be better if you changed it to be more like what Alejandro suggested on Nov 07 ($ oozie-setup.sh [prepare-war|create-db|upgrade-db|create-sharelib|upgrade-sharelib]) and that giving no arguments would print out the usage info.
        7) It would be nice if the script/java could accept either a folder or the sharelib tar archive (and automatically expand it)

        Show
        Robert Kanter added a comment - Looks better: A few things: 1) Make sure to generate the patch without the preceding a/ and b/ (--no-prefix for git) 2) Some of the indenting is off in OozieFSCLI#run ; make sure to use spaces, not tabs 3) The usage information printed out by oozie-setup.sh should mention the -sharelib argument 4) Update the documentation; make sure to mention that (by default) it should be run as the user who is going to be running the Oozie server 5) The sharelib location is actually configurable in oozie-site.xml; OozieFSCLI should read oozie-site (see OozieDBCLI for how its done there via Services ) and use the value of the oozie.service.WorkflowAppService.system.libpath property for the destination path 6) Right now, when you run the -sharelib argument, it will prepare the war file, even if the user already did that. I think it would be better if you changed it to be more like what Alejandro suggested on Nov 07 ($ oozie-setup.sh [prepare-war|create-db|upgrade-db|create-sharelib|upgrade-sharelib] ) and that giving no arguments would print out the usage info. 7) It would be nice if the script/java could accept either a folder or the sharelib tar archive (and automatically expand it)
        Hide
        Robert Kanter added a comment -

        Your latest patch only has the changes for oozie-setup.sh; the OozieFSCLI is missing. Can you reupload?

        Also, you don't need to delete the old patch when uploading a new one; JIRA allows them to have the same filename and will mark the newest one with a different color (and there's a timestamp). Its useful to be able to see what changed between versions of a patch.

        Show
        Robert Kanter added a comment - Your latest patch only has the changes for oozie-setup.sh; the OozieFSCLI is missing. Can you reupload? Also, you don't need to delete the old patch when uploading a new one; JIRA allows them to have the same filename and will mark the newest one with a different color (and there's a timestamp). Its useful to be able to see what changed between versions of a patch.
        Hide
        Bowen Zhang added a comment -

        Hi Robert, I reuploaded my patch. Sorry about that.

        Show
        Bowen Zhang added a comment - Hi Robert, I reuploaded my patch. Sorry about that.
        Hide
        Robert Kanter added a comment -

        No problem.

        Here's my feedback from your latest patch:

        1) Now that it prints the usage when no arguments are given; the line in the usage that says echo " (without options does default setup, without the Oozie webconsole)" should be changed to something like echo " (without options prints this usage information)"
        2) The variable OOZIEDB_OPTS should be named something like OOZIEFSCLI_OPTS to be more clear
        3) Line 30 in OozieFSCLI is indented too much
        4) Please add the Apache License header to the top of OozieFSCLI; you can just copy it from OozieDBCLI
        5) Line 27 in oozie-setup is longer than 132 characters
        6) If you give no arguments, it will print out INFO: Oozie webconsole disabled, ExtJS library not specified and then no arguments given and then the usage. If its missing the arguments, it shouldn't print out that first message. It also shouldn't print that out at all when doing only the sharelib
        7) When doing the sharelib, it should also print out where the destination path
        8) When you give it a folder instead of the tar for the sharelib, it uploads to share/lib/lib/ instead of share/lib/
        9) What argument do you give to make it prepare the war file? This should also be in the usage info.
        10) Please update the "Oozie Quick Start" (DG_QuickStart.twiki) and "Oozie Installation and Configuration" (AG_Install.twiki) doc pages. They're in the docs/src/site/twiki/ folder.

        Show
        Robert Kanter added a comment - No problem. Here's my feedback from your latest patch: 1) Now that it prints the usage when no arguments are given; the line in the usage that says echo " (without options does default setup, without the Oozie webconsole)" should be changed to something like echo " (without options prints this usage information)" 2) The variable OOZIEDB_OPTS should be named something like OOZIEFSCLI_OPTS to be more clear 3) Line 30 in OozieFSCLI is indented too much 4) Please add the Apache License header to the top of OozieFSCLI ; you can just copy it from OozieDBCLI 5) Line 27 in oozie-setup is longer than 132 characters 6) If you give no arguments, it will print out INFO: Oozie webconsole disabled, ExtJS library not specified and then no arguments given and then the usage. If its missing the arguments, it shouldn't print out that first message. It also shouldn't print that out at all when doing only the sharelib 7) When doing the sharelib, it should also print out where the destination path 8) When you give it a folder instead of the tar for the sharelib, it uploads to share/lib/lib/ instead of share/lib/ 9) What argument do you give to make it prepare the war file? This should also be in the usage info. 10) Please update the "Oozie Quick Start" ( DG_QuickStart.twiki ) and "Oozie Installation and Configuration" ( AG_Install.twiki ) doc pages. They're in the docs/src/site/twiki/ folder.
        Hide
        Bowen Zhang added a comment -

        I uploaded a new patch.

        Show
        Bowen Zhang added a comment - I uploaded a new patch.
        Hide
        Robert Kanter added a comment -

        That looks good to me +1
        Thanks for making all of those changes.

        Though I'd like to wait for a second opinion before committing it just to make sure there isn't anything else that we should do here.

        Show
        Robert Kanter added a comment - That looks good to me +1 Thanks for making all of those changes. Though I'd like to wait for a second opinion before committing it just to make sure there isn't anything else that we should do here.
        Hide
        Alejandro Abdelnur added a comment -

        Sorry for the late review, a few things:

        • OozieFSCLI should be renamed to OozieSharelibCLI
        • OozieSharelibCLI should support have an '-upgrade' option, without this option, if the sharelib exists in HDFS, it should fail. With the option, if the sharelib exists in HDFS, it should first delete the old version and then copy the new one.
        • OozieSharelibCLI should find out the HDFS URI from the Oozie configuration
        • oozie-setup.sh:
          • tar.gz is not a zip file, comment 'unzipping sharelib is not correct'
          • -sharelib option should take a single parameter, the local sharelib location (tar.gz or expanded)
          • help message for -sharelib should be something along the following lines:
          • the classpath construction can be simplified to be OOZIECPPATH="$ {BASEDIR}/libtools/'*':${BASEDIR}

            /libext/'*'"

        Show
        Alejandro Abdelnur added a comment - Sorry for the late review, a few things: OozieFSCLI should be renamed to OozieSharelibCLI OozieSharelibCLI should support have an '-upgrade' option, without this option, if the sharelib exists in HDFS, it should fail. With the option, if the sharelib exists in HDFS, it should first delete the old version and then copy the new one. OozieSharelibCLI should find out the HDFS URI from the Oozie configuration oozie-setup.sh: tar.gz is not a zip file, comment 'unzipping sharelib is not correct' -sharelib option should take a single parameter, the local sharelib location (tar.gz or expanded) help message for -sharelib should be something along the following lines: the classpath construction can be simplified to be OOZIECPPATH="$ {BASEDIR}/libtools/'*':${BASEDIR} /libext/'*'"
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro, are you done with your third comment under oozie-setup.sh about the help message?

        Show
        Bowen Zhang added a comment - Hi Alejandro, are you done with your third comment under oozie-setup.sh about the help message?
        Hide
        Alejandro Abdelnur added a comment -

        Sorry, forgot that one, I've meant to say something like 'installs or upgrades Oozie sharelib'.

        Show
        Alejandro Abdelnur added a comment - Sorry, forgot that one, I've meant to say something like 'installs or upgrades Oozie sharelib'.
        Hide
        Alejandro Abdelnur added a comment -

        And as a follow up thought .... (not part of this JIRA). We should think how to consolidate all setup operations in the oozie-setup.sh script, something like:

        oozie-setup.sh TASK [OPTIONS]
        
        Where the avail tasks are:
        
          * war
          * db [-create|-updgrade] [-run] [-sqlfile FILE]
          * sharelib [-install|-upgrade]
        
        Show
        Alejandro Abdelnur added a comment - And as a follow up thought .... (not part of this JIRA). We should think how to consolidate all setup operations in the oozie-setup.sh script, something like: oozie-setup.sh TASK [OPTIONS] Where the avail tasks are: * war * db [-create|-updgrade] [-run] [-sqlfile FILE] * sharelib [-install|-upgrade]
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro, I doubt we can find HDFS URI from the Oozie configuration since the hadoop-conf/core-site.xml file inside oozie conf directory doesn't contain the property "fs.default.name".

        Show
        Bowen Zhang added a comment - Hi Alejandro, I doubt we can find HDFS URI from the Oozie configuration since the hadoop-conf/core-site.xml file inside oozie conf directory doesn't contain the property "fs.default.name".
        Hide
        Bowen Zhang added a comment -

        Or we can ask the user to specify their hadoop_home directory to get hdfs uri.

        Show
        Bowen Zhang added a comment - Or we can ask the user to specify their hadoop_home directory to get hdfs uri.
        Hide
        Alejandro Abdelnur added a comment -

        Bowen, you have a point, I was thinking about use the default hadoop-conf in oozie (the one with '*'), but you are right, we have to specify the FileSystem URI. Still the OozieSharelibCLI would use the HadoopAccessor to create it.

        Thx

        Show
        Alejandro Abdelnur added a comment - Bowen, you have a point, I was thinking about use the default hadoop-conf in oozie (the one with '*'), but you are right, we have to specify the FileSystem URI. Still the OozieSharelibCLI would use the HadoopAccessor to create it. Thx
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro, I uploaded a new patch and BTW, your suggestion of OOZIECPPATH construction doesn't work for some reason.

        Show
        Bowen Zhang added a comment - Hi Alejandro, I uploaded a new patch and BTW, your suggestion of OOZIECPPATH construction doesn't work for some reason.
        Hide
        Alejandro Abdelnur added a comment -

        Bowen, how exactly are your constructing the CP that is not working, you should do something like (from Hadoop's hadoop-config.sh):

        CLASSPATH=$

        {CLASSPATH}

        :$HADOOP_COMMON_HOME/$HADOOP_COMMON_LIB_JARS_DIR'/*'

        Show
        Alejandro Abdelnur added a comment - Bowen, how exactly are your constructing the CP that is not working, you should do something like (from Hadoop's hadoop-config.sh): CLASSPATH=$ {CLASSPATH} :$HADOOP_COMMON_HOME/$HADOOP_COMMON_LIB_JARS_DIR'/*'
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro, I think the issue is there shouldn't be double quotes for your path construction. Now, I resolved the issue.

        Show
        Bowen Zhang added a comment - Hi Alejandro, I think the issue is there shouldn't be double quotes for your path construction. Now, I resolved the issue.
        Hide
        Robert Kanter added a comment -

        Hi Bowen,

        Instead of doing this to get the FileSystem:

        Configuration config = new Configuration();
        config.set("fs.default.name", args[2]);
        FileSystem fs = FileSystem.get(config);
        Path srcPath = new Path(args[1]);
        

        you should use the HadoopAccessorService because it'll load in other configurations that may be needed . Something like this (please double check that this code works, I haven't actually tried it):

        Path srcPath = new Path(args[1]);
        URI srcUri = srcPath.toUri();
        HadoopAccessorService has = Services.get().get(HadoopAccessorService.class);
        Configuration fsConf = has.createJobConf(uri.getAuthority());
        FileSystem fs = has.createFileSystem(System.getenv("user.name"), uri, fsConf);
        

        Also, when built, the sharelib tar.gz is always named oozie-sharelib-${version}.tar.gz and located in the root dir. Can you make it so that the createsharelib and upgradesharelib options don't require the SHARED_LIBRARY argument? (if not provided, it would try to use oozie-sharelib-${version}.tar.gz).
        i.e

        [-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]]
        [-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]]
        
        Show
        Robert Kanter added a comment - Hi Bowen, Instead of doing this to get the FileSystem : Configuration config = new Configuration(); config.set( "fs. default .name" , args[2]); FileSystem fs = FileSystem.get(config); Path srcPath = new Path(args[1]); you should use the HadoopAccessorService because it'll load in other configurations that may be needed . Something like this (please double check that this code works, I haven't actually tried it): Path srcPath = new Path(args[1]); URI srcUri = srcPath.toUri(); HadoopAccessorService has = Services.get().get(HadoopAccessorService.class); Configuration fsConf = has.createJobConf(uri.getAuthority()); FileSystem fs = has.createFileSystem( System .getenv( "user.name" ), uri, fsConf); Also, when built, the sharelib tar.gz is always named oozie-sharelib-${version}.tar.gz and located in the root dir. Can you make it so that the createsharelib and upgradesharelib options don't require the SHARED_LIBRARY argument? (if not provided, it would try to use oozie-sharelib-${version}.tar.gz ). i.e [-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]] [-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]]
        Hide
        Bowen Zhang added a comment -

        I made a latest patch and for the OozieSharelibCLI.java, I didn't use hadoopAccessorService for creating file system since its file system doesn't support local file format.

        Show
        Bowen Zhang added a comment - I made a latest patch and for the OozieSharelibCLI.java, I didn't use hadoopAccessorService for creating file system since its file system doesn't support local file format.
        Hide
        Robert Kanter added a comment -

        One last thing that I should have mentioned earlier: we should start up the two Services properly; that is, something like this:

        Services services = new Services();
        services.getConf().set(Services.CONF_SERVICE_CLASSES, HadoopAccessorService.class, LiteWorkflowAppService.class);
        services.init();
        LiteWorkflowAppService lwas = services.get(LiteWorkflowAppService.class);
        HadoopAccessorService has = services.get(HadoopAccessorService.class);
        //do stuff
        services.destroy();
        
        Show
        Robert Kanter added a comment - One last thing that I should have mentioned earlier: we should start up the two Services properly; that is, something like this: Services services = new Services(); services.getConf().set(Services.CONF_SERVICE_CLASSES, HadoopAccessorService.class, LiteWorkflowAppService.class); services.init(); LiteWorkflowAppService lwas = services.get(LiteWorkflowAppService.class); HadoopAccessorService has = services.get(HadoopAccessorService.class); // do stuff services.destroy();
        Hide
        Robert Kanter added a comment -

        The Services stuff looks good; thanks.

        When I tried running it without specifying the sharelib tar (i.e. use the default), I got this error:

        Exception in thread "main" java.io.IOException: /Users/rkanter/dev/oozie/oozie-sharelib-.tar.gz/lib cannot be found
        	at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:73)
        	at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:35)
        

        The name of the tar is oozie-sharelib-3.4.0-SNAPSHOT.tar.gz, so its not allowing for the version number. I believe its this part of the script:

        sharelibPath=${OOZIE_HOME##*/}
        sharelibPath=${OOZIE_HOME}/oozie-sharelib-${sharelibPath:6}.tar.gz
        

        I'm not sure about the syntax you're using there, but is that trying to grab the version number from the OOZIE_HOME path? If so, that's not going to always work because the OOZIE_HOME doesn't necessarily contain the version number (e.g if it gets deployed to a standardized location like /usr/lib/oozie/). I'm not sure of the syntax, but I recommend you just try to find a file that matches oozie-sharelib-*.tar.gz and not worry that the wildcard matches the exact version of Oozie. I think its safe to assume that the user doesn't have multiple versions of the sharelib tar; or maybe you can just choose the one with the highest version.

        Show
        Robert Kanter added a comment - The Services stuff looks good; thanks. When I tried running it without specifying the sharelib tar (i.e. use the default), I got this error: Exception in thread "main" java.io.IOException: /Users/rkanter/dev/oozie/oozie-sharelib-.tar.gz/lib cannot be found at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:73) at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:35) The name of the tar is oozie-sharelib-3.4.0-SNAPSHOT.tar.gz , so its not allowing for the version number. I believe its this part of the script: sharelibPath=${OOZIE_HOME##*/} sharelibPath=${OOZIE_HOME}/oozie-sharelib-${sharelibPath:6}.tar.gz I'm not sure about the syntax you're using there, but is that trying to grab the version number from the OOZIE_HOME path? If so, that's not going to always work because the OOZIE_HOME doesn't necessarily contain the version number (e.g if it gets deployed to a standardized location like /usr/lib/oozie/ ). I'm not sure of the syntax, but I recommend you just try to find a file that matches oozie-sharelib-*.tar.gz and not worry that the wildcard matches the exact version of Oozie. I think its safe to assume that the user doesn't have multiple versions of the sharelib tar; or maybe you can just choose the one with the highest version.
        Hide
        Robert Kanter added a comment -

        +1
        I think everything looks good; thanks for working through all of the comments.
        I'll just ask Alejandro to take another look.

        Show
        Robert Kanter added a comment - +1 I think everything looks good; thanks for working through all of the comments. I'll just ask Alejandro to take another look.
        Hide
        Alejandro Abdelnur added a comment -

        Bowen,

        Things look good. Is there any reason why you the sharelibCLI class does not use OozieCLI (like like the DB tool) to handle the params options? I think that would simplify the script and the sharelibCLI.

        Also, and I'm bringing here OOZIE-670 in scope. As we are consolidating all setup configuration in Oozie setup, we should see that options feel consistent for the different tasks.

        My thoughts on how things should be through oozie-setup.sh after everything is consolidated is:

        oozie-setup.sh prepare-war [-jars <PATHS>] [-extjs <PATH>]
                       db create -run [-sqlfile <FILE>]
                       db upgrade -run [-sqlfile <FILE>]
                       db postupgrade -run [-sqlfile <FILE>]
                       sharelib create -locallib <PATH> -fs <FS_URI>
                       sharelib upgrade -locallib <LOCAL_PATH> -fs <FS_URI>
        

        This means that the oozie-setup.sh script would do for db and sharelib is to delegate to the CLI removing the sub-command (db or sharelib).

        This means that the java CLI (db and sharelib) will have to handle the subsubcommand and options. the DB CLI already does this. So the integration of DB CLI in oozie-setup.sh would be simple. That is why I'm asking to do the same for the sharelib CLI.

        Makes sense?

        Show
        Alejandro Abdelnur added a comment - Bowen, Things look good. Is there any reason why you the sharelibCLI class does not use OozieCLI (like like the DB tool) to handle the params options? I think that would simplify the script and the sharelibCLI. Also, and I'm bringing here OOZIE-670 in scope. As we are consolidating all setup configuration in Oozie setup, we should see that options feel consistent for the different tasks. My thoughts on how things should be through oozie-setup.sh after everything is consolidated is: oozie-setup.sh prepare-war [-jars <PATHS>] [-extjs <PATH>] db create -run [-sqlfile <FILE>] db upgrade -run [-sqlfile <FILE>] db postupgrade -run [-sqlfile <FILE>] sharelib create -locallib <PATH> -fs <FS_URI> sharelib upgrade -locallib <LOCAL_PATH> -fs <FS_URI> This means that the oozie-setup.sh script would do for db and sharelib is to delegate to the CLI removing the sub-command (db or sharelib). This means that the java CLI (db and sharelib) will have to handle the subsubcommand and options. the DB CLI already does this. So the integration of DB CLI in oozie-setup.sh would be simple. That is why I'm asking to do the same for the sharelib CLI. Makes sense?
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro,
        I am not sure whether we are going to remove ooziedb.sh in the consolidation process. For the setup consolidation, i think the format for sharelib is similar to the one suggested by Robert:[-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]] and [-upgradesharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY]].
        I didn't use the command option param in my sharelibCLI is that we have at most 2 arguments in the main function as you can see from the above argument format on the command line

        Show
        Bowen Zhang added a comment - Hi Alejandro, I am not sure whether we are going to remove ooziedb.sh in the consolidation process. For the setup consolidation, i think the format for sharelib is similar to the one suggested by Robert:[-createsharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY] ] and [-upgradesharelib HDFS_DEFAULT_NAME [SHARED_LIBRARY] ]. I didn't use the command option param in my sharelibCLI is that we have at most 2 arguments in the main function as you can see from the above argument format on the command line
        Hide
        Alejandro Abdelnur added a comment -

        ooziedb has also 2 options only (-run and -sqlfile) and it is using OozieCLI.

        regarding moving ooziedb.sh into the consolidation, I think we should, and in preparation/alignment with that is that I'm proposing the abo e oozie-setup.sh CLI syntax.

        Show
        Alejandro Abdelnur added a comment - ooziedb has also 2 options only (-run and -sqlfile) and it is using OozieCLI. regarding moving ooziedb.sh into the consolidation, I think we should, and in preparation/alignment with that is that I'm proposing the abo e oozie-setup.sh CLI syntax.
        Hide
        Bowen Zhang added a comment -

        Since these two are separate jira tickets, would you consider submitting this one first before I work on the merge and consolidation in the other ticket?

        Show
        Bowen Zhang added a comment - Since these two are separate jira tickets, would you consider submitting this one first before I work on the merge and consolidation in the other ticket?
        Hide
        Alejandro Abdelnur added a comment -

        Sure, just that we have the args/options already aligned with the aimed end result, thus my prev feedback.

        Show
        Alejandro Abdelnur added a comment - Sure, just that we have the args/options already aligned with the aimed end result, thus my prev feedback.
        Hide
        Alejandro Abdelnur added a comment -

        Bowen,

        Mmhh, it seems I was not explaining things clearly, when using OozieCLI to conform to the syntax above, it should be something like:

            HELP_CMD = "help";
           CREATE_CMD = "create";
           UPGRADE_CMD = "upgrade;
           LIB_OPT = "locallib";
           FS_OPT = "fs";
        
           protected Options createUpgradeOptions(String subCommand) {
                Option sharelib = new Option(LIB_OPT, true, "Local share library Oozie tar.gz file (or expanded directory) to " + subCommand);
                Option uri = new Option(FS_OPT, true, "URI of the FileSystem to " + subCommand + " Oozie share library");
                Options options = new Options();
                options.addOption(sharelib);
                options.addOption(uri);
                return options;
            }
        
            ...
        
                CLIParser parser = new CLIParser("ooziedb.sh", HELP_INFO);
                parser.addCommand(HELP_CMD, "", "display usage for all commands or specified command", new Options(), false);
                parser.addCommand(CREATE_CMD, "", "create Oozie sharelib", createUpgradeOptions(CREATE_CMD), false);
                parser.addCommand(UPGRADE_CMD, "", "upgrade Oozie sharelib", createUpgradeOptions(UPGRADE_CMD), false);
        

        Thx

        Show
        Alejandro Abdelnur added a comment - Bowen, Mmhh, it seems I was not explaining things clearly, when using OozieCLI to conform to the syntax above, it should be something like: HELP_CMD = "help" ; CREATE_CMD = "create" ; UPGRADE_CMD = "upgrade; LIB_OPT = "locallib" ; FS_OPT = "fs" ; protected Options createUpgradeOptions( String subCommand) { Option sharelib = new Option(LIB_OPT, true , "Local share library Oozie tar.gz file (or expanded directory) to " + subCommand); Option uri = new Option(FS_OPT, true , "URI of the FileSystem to " + subCommand + " Oozie share library" ); Options options = new Options(); options.addOption(sharelib); options.addOption(uri); return options; } ... CLIParser parser = new CLIParser( "ooziedb.sh" , HELP_INFO); parser.addCommand(HELP_CMD, "", " display usage for all commands or specified command", new Options(), false ); parser.addCommand(CREATE_CMD, "", " create Oozie sharelib", createUpgradeOptions(CREATE_CMD), false ); parser.addCommand(UPGRADE_CMD, "", " upgrade Oozie sharelib", createUpgradeOptions(UPGRADE_CMD), false ); Thx
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro,
        I think the difference is in my format and also Robert's format, I expect users to type "oozie-setup.sh -createsharelib hdfs sharelibLocation" so I have "-createsharelib" and "-upgradesharelib" command. In your format, you expect users to type "oozie-setup.sh sharelib create -locallib <PATH> -fs <FS_URI>", therefore you have "create" and "upgrade" as command. I think my way resembles more of the current format for users since they are typing "oozie-setup.sh -hadoop arg1 arg2 -extjs arg -jars arg". What's your take?

        Show
        Bowen Zhang added a comment - Hi Alejandro, I think the difference is in my format and also Robert's format, I expect users to type "oozie-setup.sh -createsharelib hdfs sharelibLocation" so I have "-createsharelib" and "-upgradesharelib" command. In your format, you expect users to type "oozie-setup.sh sharelib create -locallib <PATH> -fs <FS_URI>", therefore you have "create" and "upgrade" as command. I think my way resembles more of the current format for users since they are typing "oozie-setup.sh -hadoop arg1 arg2 -extjs arg -jars arg". What's your take?
        Hide
        Alejandro Abdelnur added a comment -

        following oozie CLI, hadoop CLI, hadoop admin CLIs, git CLI, the first thing after a CLI command is a subcommand (not an option), in the case of oozie-setup after OOZIE-670 is done it would look like the crafted help in one my previous comments. Because of that I'm suggesting to do that work for the the sharelib already.

        Note that the db and sharelib, in the proposed new layout, have a subsubcommand. This follows a common practice done in GIT for more complex tasks (ie git remote ...). And is a minimal change to the existing format of the ooziedb.sh, where 'ooziedb.sh' is replaced with 'oozie-setup.sh db', all the rest is the same.

        The tweaks I'm suggesting for sharelib is to align/be-consistent on how Oozie DB setup command line will be.

        Show
        Alejandro Abdelnur added a comment - following oozie CLI, hadoop CLI, hadoop admin CLIs, git CLI, the first thing after a CLI command is a subcommand (not an option), in the case of oozie-setup after OOZIE-670 is done it would look like the crafted help in one my previous comments. Because of that I'm suggesting to do that work for the the sharelib already. Note that the db and sharelib, in the proposed new layout, have a subsubcommand. This follows a common practice done in GIT for more complex tasks (ie git remote ...). And is a minimal change to the existing format of the ooziedb.sh, where 'ooziedb.sh' is replaced with 'oozie-setup.sh db', all the rest is the same. The tweaks I'm suggesting for sharelib is to align/be-consistent on how Oozie DB setup command line will be.
        Hide
        Bowen Zhang added a comment -

        If this is the case, then it will require a change in setup.sh as to how it will loop through and parse all the command arguments

        Show
        Bowen Zhang added a comment - If this is the case, then it will require a change in setup.sh as to how it will loop through and parse all the command arguments
        Hide
        Alejandro Abdelnur added a comment -

        I've just done a build with the patch applied:

        • oozie-setup.sh should parse the 'sharelib' subcommand only, after that delegate the rest of the parsing to the OozieSharelibCLI class
        • oozie-setup.sh help is not in synch with the new syntax
        • the sharelib create should fail if the sharelib dir already exists in HDFS
        • the sharelib upgrade should fail if the sharelib dir does not exist in HDFS
        Show
        Alejandro Abdelnur added a comment - I've just done a build with the patch applied: oozie-setup.sh should parse the 'sharelib' subcommand only, after that delegate the rest of the parsing to the OozieSharelibCLI class oozie-setup.sh help is not in synch with the new syntax the sharelib create should fail if the sharelib dir already exists in HDFS the sharelib upgrade should fail if the sharelib dir does not exist in HDFS
        Hide
        Bowen Zhang added a comment -

        I don't think we should delegate the parsing to the OozieSharelibCLI like the way in OozieDBCLI.
        If the user types: oozie-setup.sh prepare-war -hadooop version hadoop_home -extjs path sharelib create -fs uri -locallib path, are we trying to jam all these command line stuff to OozieSharelibCLI via "$

        {@}

        "? This is adding unnecessary complexities into OozieSharelibCLI.
        And if the user types: oozie-setup.sh prepare-war -hadooop version hadoop_home -extjs path sharelib create -fs uri, then are we letting the java class to figure out the default sharelib.tar.gz?
        BTW, when I type oozie-setup.sh help, it will indeed print out the usage info, isn't it what we want?

        Show
        Bowen Zhang added a comment - I don't think we should delegate the parsing to the OozieSharelibCLI like the way in OozieDBCLI. If the user types: oozie-setup.sh prepare-war -hadooop version hadoop_home -extjs path sharelib create -fs uri -locallib path, are we trying to jam all these command line stuff to OozieSharelibCLI via "$ {@} "? This is adding unnecessary complexities into OozieSharelibCLI. And if the user types: oozie-setup.sh prepare-war -hadooop version hadoop_home -extjs path sharelib create -fs uri, then are we letting the java class to figure out the default sharelib.tar.gz? BTW, when I type oozie-setup.sh help, it will indeed print out the usage info, isn't it what we want?
        Hide
        Alejandro Abdelnur added a comment -

        Bowen, what I've meant is to do minimal filtering/redirection logic in oozie-setup.sh, something like:

        ...
        subcommand=$1
        # lets get rid of the first argument
        shift
        swich "$subcommand" in
          prepare-war)
            #PREPARE WAR LOGIC
          db)
            # setup ENV
            # call OozieDBCLI "${@}"
            ;;
          sharelib)
            # setup ENV
            # call OozieSharelibCLI "${@}"
            ;;
          *)
            # PRINT TOP LEVEL HELP, MENTIONING that each subcommand supports 'help' for more detailed info about the command
            ;;
        esac
        
        Show
        Alejandro Abdelnur added a comment - Bowen, what I've meant is to do minimal filtering/redirection logic in oozie-setup.sh, something like: ... subcommand=$1 # lets get rid of the first argument shift swich "$subcommand" in prepare-war) #PREPARE WAR LOGIC db) # setup ENV # call OozieDBCLI "${@}" ;; sharelib) # setup ENV # call OozieSharelibCLI "${@}" ;; *) # PRINT TOP LEVEL HELP, MENTIONING that each subcommand supports 'help' for more detailed info about the command ;; esac
        Hide
        Robert Kanter added a comment -

        I'm also in favor of minimal logic in oozie-setup.sh; its currently a little hard to follow. Simplifying it will also make it easier to maintain/update in the future.

        Show
        Robert Kanter added a comment - I'm also in favor of minimal logic in oozie-setup.sh; its currently a little hard to follow. Simplifying it will also make it easier to maintain/update in the future.
        Hide
        Alejandro Abdelnur added a comment -

        just to be clear, don't want to clean up completely the oozie-setup.sh here, just prepare things for the follow up JIRA.

        Show
        Alejandro Abdelnur added a comment - just to be clear, don't want to clean up completely the oozie-setup.sh here, just prepare things for the follow up JIRA.
        Hide
        Bowen Zhang added a comment -

        Hi Alejandro, the issue with your code is that user cannot prepare war files and upload sharelib in the same run of oozie-setup.sh. Since you are using "$

        {@}

        ", you are jamming all the possible commands to the java class which was created with the intent to just do one thing.
        My point is: if we allow users to create sharelib, prepare war files, and create db in one single setup.sh run, and we keep the current functionality of each java class without creating unnecessary parsing, then we need to parse all the subcommand and option values in setup.sh. Your above code works well when user is allowed to perform only one command in each run of setup.sh.

        Show
        Bowen Zhang added a comment - Hi Alejandro, the issue with your code is that user cannot prepare war files and upload sharelib in the same run of oozie-setup.sh. Since you are using "$ {@} ", you are jamming all the possible commands to the java class which was created with the intent to just do one thing. My point is: if we allow users to create sharelib, prepare war files, and create db in one single setup.sh run, and we keep the current functionality of each java class without creating unnecessary parsing, then we need to parse all the subcommand and option values in setup.sh. Your above code works well when user is allowed to perform only one command in each run of setup.sh.
        Hide
        Alejandro Abdelnur added a comment -

        you are correct, but I don't see that as a limitation, that is how CLIs with subcommands work, you do one thing at the time.

        Show
        Alejandro Abdelnur added a comment - you are correct, but I don't see that as a limitation, that is how CLIs with subcommands work, you do one thing at the time.
        Hide
        Alejandro Abdelnur added a comment -

        Hi Bowen, are you following up on this? I think this is a great improvement and I'd like to get it in ASAP. Thx

        Show
        Alejandro Abdelnur added a comment - Hi Bowen, are you following up on this? I think this is a great improvement and I'd like to get it in ASAP. Thx
        Hide
        Hadoop QA added a comment -

        Testing JIRA OOZIE-1054

        Cleaning local svn workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . -1 the patch contains 2 line(s) longer than 132 characters
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        +1 TESTS
        . Tests run: 951
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/327/

        Show
        Hadoop QA added a comment - Testing JIRA OOZIE-1054 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . -1 the patch contains 2 line(s) longer than 132 characters . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 951 +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/327/
        Hide
        Hadoop QA added a comment -

        Testing JIRA OOZIE-1054

        Cleaning local svn workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . -1 the patch contains 2 line(s) longer than 132 characters
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        +1 TESTS
        . Tests run: 951
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/328/

        Show
        Hadoop QA added a comment - Testing JIRA OOZIE-1054 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . -1 the patch contains 2 line(s) longer than 132 characters . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 951 +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/328/
        Hide
        Alejandro Abdelnur added a comment -

        Bowen, it looks good, just a few final minor things and it is ready to go. THANKS, nice job.

        • oozie-setup.sh, help message. minor change: SHARED_LIBRARY path to the Oozie sharelib to install, it can the tarball or an expanded version of it. If omitted, the Oozie sahrelib tarball from the Oozie installation directory will be used.
        • oozie-setup.sh, help message. minor change: fails if there is no existing sharelib installed in HDFS.
        • the above corrections should go in the AG_Install twiki page as well.
        • the DG_QuickStart twiki should be updated accordingly as well.
        • creating a temp dir, we should use the system tmp dir, the trick for doing that is:
          tempDir = File.createTempFile("oozie", ".dir");
          tempDir.delete();
          tempDir.mkdir();
          tempDir.deleteOnExit();
        
        • the help does not work correctly:
        $ bin/oozie-setup.sh sharelib help
          setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
        
        Error: -fs option must be specified
        ....
        
        Show
        Alejandro Abdelnur added a comment - Bowen, it looks good, just a few final minor things and it is ready to go. THANKS, nice job. oozie-setup.sh, help message. minor change: SHARED_LIBRARY path to the Oozie sharelib to install, it can the tarball or an expanded version of it. If omitted, the Oozie sahrelib tarball from the Oozie installation directory will be used. oozie-setup.sh, help message. minor change: fails if there is no existing sharelib installed in HDFS. the above corrections should go in the AG_Install twiki page as well. the DG_QuickStart twiki should be updated accordingly as well. creating a temp dir, we should use the system tmp dir, the trick for doing that is: tempDir = File.createTempFile( "oozie" , ".dir" ); tempDir.delete(); tempDir.mkdir(); tempDir.deleteOnExit(); the help does not work correctly: $ bin/oozie-setup.sh sharelib help setting CATALINA_OPTS= "$CATALINA_OPTS -Xmx1024m" Error: -fs option must be specified ....
        Hide
        Alejandro Abdelnur added a comment -

        forgot to add that I've done a build with the patch and tested the sharelib create/upgrade commands successfully.

        Show
        Alejandro Abdelnur added a comment - forgot to add that I've done a build with the patch and tested the sharelib create/upgrade commands successfully.
        Hide
        Alejandro Abdelnur added a comment -

        +1 pending jenkins. tested sharebli create/upgrade/help commands successfully.

        Show
        Alejandro Abdelnur added a comment - +1 pending jenkins. tested sharebli create/upgrade/help commands successfully.
        Hide
        Hadoop QA added a comment -

        Testing JIRA OOZIE-1054

        Cleaning local svn workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . -1 the patch contains 1 line(s) longer than 132 characters
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        -1 TESTS
        . Tests run: 951
        . Tests failed: 0
        . Tests errors: 1

        . The patch failed the following testcases:

        .

        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/330/

        Show
        Hadoop QA added a comment - Testing JIRA OOZIE-1054 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . -1 the patch contains 1 line(s) longer than 132 characters . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 951 . Tests failed: 0 . Tests errors: 1 . The patch failed the following testcases: . +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/330/
        Hide
        Hadoop QA added a comment -

        Testing JIRA OOZIE-1054

        Cleaning local svn workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . -1 the patch contains 1 line(s) longer than 132 characters
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        +1 TESTS
        . Tests run: 951
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/331/

        Show
        Hadoop QA added a comment - Testing JIRA OOZIE-1054 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . -1 the patch contains 1 line(s) longer than 132 characters . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 951 +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/331/
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Bowen. Committed to trunk.

        Show
        Alejandro Abdelnur added a comment - Thanks Bowen. Committed to trunk.
        Hide
        Robert Kanter added a comment -

        Closing issue; Oozie 3.3.2 is released

        Show
        Robert Kanter added a comment - Closing issue; Oozie 3.3.2 is released

          People

          • Assignee:
            Bowen Zhang
            Reporter:
            Robert Kanter
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development