Chukwa
  1. Chukwa
  2. CHUKWA-288

Cleaner build management for chukwa-pig.jar

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.2.0
    • Fix Version/s: None
    • Component/s: Build and Test Code
    • Labels:
      None
    • Environment:

      Redhat EL 5.1, Java 6

      Description

      The current build system generates chukwa-pig.jar in CHUKWA_HOME/contrib/chukwa-pig. This should be changed to CHUKWA_HOME/contrib/chukwa-pig/build. The change would prevent mangling of source file and binary file.

        Activity

        Hide
        Jerome Boulon added a comment -

        No, current build generate everything inside CHUKWA_HOME/contrib/chukwa-pig/build then copy the jar file to CHUKWA_HOME/contrib/chukwa-pig the same way as pig/hadoop.

        And Chukwa should be change to have the same thing. we shouldn't have to move to the build directory to get the jar files.
        The expectation is to:

        • download chukwa,
        • do ant
        • and run it right away from current directory,
          everything else will just not work.

        As a user I don't have to read a quick start page to learn that I have to go to build/chukwa-xxxx/ to run it.

        Show
        Jerome Boulon added a comment - No, current build generate everything inside CHUKWA_HOME/contrib/chukwa-pig/build then copy the jar file to CHUKWA_HOME/contrib/chukwa-pig the same way as pig/hadoop. And Chukwa should be change to have the same thing. we shouldn't have to move to the build directory to get the jar files. The expectation is to: download chukwa, do ant and run it right away from current directory, everything else will just not work. As a user I don't have to read a quick start page to learn that I have to go to build/chukwa-xxxx/ to run it.
        Hide
        Ari Rabkin added a comment -

        +1 to having ant make Chukwa runnable immediately.

        I'm open to saying "the chukwa scripts should assume jars live in /build", by the way, as an alternative to moving them to CHUKWA_HOME after the build.

        Show
        Ari Rabkin added a comment - +1 to having ant make Chukwa runnable immediately. I'm open to saying "the chukwa scripts should assume jars live in /build", by the way, as an alternative to moving them to CHUKWA_HOME after the build.
        Hide
        Eric Yang added a comment -

        Looked at the behavior of hadoop, their jar files are built in build directory and not copied to the top level source code directory.
        I think it would be more sensible to compile and put in build directory, and ensure that after calling ant,
        the build directory could be acting as CHUKWA_HOME.

        Show
        Eric Yang added a comment - Looked at the behavior of hadoop, their jar files are built in build directory and not copied to the top level source code directory. I think it would be more sensible to compile and put in build directory, and ensure that after calling ant, the build directory could be acting as CHUKWA_HOME.
        Hide
        Jerome Boulon added a comment -

        I personally don't like the idea of always running from the build directory.
        The build directory contains a lot of temporary files (classes, test cases,...), I would rather prefer to 2 configurations.

        The "Dev" configuration, by default. All scripts runs out of build/classes and/or build/contrib/<subProject>/classes.
        The "Prod" configuration, you type ant dist or ant package, the dist directory is populated with new jars (based on build/classes), build directory is deleted. Scripts will now use jars from the dist directory. There's a hierarchy inside the dist directory:

        CHUKWA_HOME/dist/ <Chukwa's jars>
        CHUKWA_HOME/dist//contrib/<SubProject>/<subProject's jars>

        External jars should come from CHUKWA_HOME/lib/ or CHUKWA_HOME/contrib/<SubProject>/lib/
        conf could also be renamed to template and the real configuration will always be generated in build/conf and moved to dist/conf for the "Production" mode
        There's no longer a chukwa-xxx inside the build directory.

        Show
        Jerome Boulon added a comment - I personally don't like the idea of always running from the build directory. The build directory contains a lot of temporary files (classes, test cases,...), I would rather prefer to 2 configurations. The "Dev" configuration, by default. All scripts runs out of build/classes and/or build/contrib/<subProject>/classes. The "Prod" configuration, you type ant dist or ant package, the dist directory is populated with new jars (based on build/classes), build directory is deleted. Scripts will now use jars from the dist directory. There's a hierarchy inside the dist directory: CHUKWA_HOME/dist/ <Chukwa's jars> CHUKWA_HOME/dist//contrib/<SubProject>/<subProject's jars> External jars should come from CHUKWA_HOME/lib/ or CHUKWA_HOME/contrib/<SubProject>/lib/ conf could also be renamed to template and the real configuration will always be generated in build/conf and moved to dist/conf for the "Production" mode There's no longer a chukwa-xxx inside the build directory.
        Hide
        Ari Rabkin added a comment -

        This seems like the sort of thing where there's probably a Right Answer. I would vote to ask some of the Hadoop core team for advice.

        Show
        Ari Rabkin added a comment - This seems like the sort of thing where there's probably a Right Answer. I would vote to ask some of the Hadoop core team for advice.
        Hide
        Eric Yang added a comment -

        0.5.0 has been released, move to future release.

        Show
        Eric Yang added a comment - 0.5.0 has been released, move to future release.

          People

          • Assignee:
            Jerome Boulon
            Reporter:
            Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development