Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: None

      Description

      Kafka has very simple build requirements and a system like Ant is well suited for a clean and concise build. I have an experimental patch that does just this - replaces SBT with Ant+Ivy. IMO, this approach is cleaner, clearer, and more developer friendly.

      Dependencies are localized to one directory in the project rather than living in ~/.ivy2 and elsewhere. This makes manual classpath building very simple (just one glob) and also makes packaging the libs very easy.

      Testing is done through junit rather than scalatest. The Kafka tests use `org.scalatest.junit.JUnitSuite` which allow the tests to be executed through the junit test runner.

      Management of the Scala version is handled through Ivy. The way I have laid out the Ant script, the Scala version can be changed by setting a different runtime property (-Dscala.version=2.8.2). Cross-compilation of the Kafka artifact would be simple to add.

      The one downside to this approach is lack of an incremental build. `scalac` is deprecating its incremental build capabilities in coming versions. The suggested solution to this is to use an IDE that supports incremental builds.

      The main motivation for this approach, to me at least, is that a developer can look at build.xml and immediately understand what is going on (with the exception maybe of the <ivy: .../> actions which would not be changing). This is largely not true for SBT unless someone is already familiar with SBT.

        Activity

        David Arthur created issue -
        Hide
        David Arthur added a comment -

        Attaching initial patch set

        Show
        David Arthur added a comment - Attaching initial patch set
        David Arthur made changes -
        Field Original Value New Value
        Attachment 0001-Compile-and-package-Kafka-with-Ant-Ivy.patch [ 12577522 ]
        Attachment 0002-Tests-all-passing.patch [ 12577523 ]
        Attachment 0003-Parameterize-scala-version-add-offline-mode.patch [ 12577524 ]
        Hide
        David Arthur added a comment -

        Is everyone happy with the current build (SBT), or is there still interest in creating an Ant/Ivy build?

        Show
        David Arthur added a comment - Is everyone happy with the current build (SBT), or is there still interest in creating an Ant/Ivy build?
        Hide
        Joe Stein added a comment -

        I think build and deployment is definitely something to address more in 0.9

        In regards to SBT it has classically has moved fast and resourcing through shared knowledge but has improved some.

        I don't think having an Ant+Ivy way of doing it also is bad just some more updating maybe we could start using Gradle http://www.gradle.org/docs/current/dsl/index.html some? Haven't used it myself but looking to jump into it some.

        That reminds me SBT is up to 0.13 we shouldn't jump yet to that until we game plan some

        Show
        Joe Stein added a comment - I think build and deployment is definitely something to address more in 0.9 In regards to SBT it has classically has moved fast and resourcing through shared knowledge but has improved some. I don't think having an Ant+Ivy way of doing it also is bad just some more updating maybe we could start using Gradle http://www.gradle.org/docs/current/dsl/index.html some? Haven't used it myself but looking to jump into it some. That reminds me SBT is up to 0.13 we shouldn't jump yet to that until we game plan some
        Hide
        David Arthur added a comment -

        Gradle is definitely another one worth investigating (first class support for Scala is nice). My main thing is I like the build to be easily debuggable. Whenever you have some build DSL that's predefined by a framework like Gradle/SBT/Maven, it can make things hard to debug.

        IMO, Ant+Ivy is a lowest common denominator type of thing. Most people coming from the Java world know Ant, and it is mostly self-descriptive. Ivy is what SBT and Gradle use under the hood for dependency management and isn't too difficult to use on its own (there is some boilerplate involved).

        I think the main thing is we pick something all the devs are comfortable with.

        That said, Gradle does looks pretty nice

        Show
        David Arthur added a comment - Gradle is definitely another one worth investigating (first class support for Scala is nice). My main thing is I like the build to be easily debuggable. Whenever you have some build DSL that's predefined by a framework like Gradle/SBT/Maven, it can make things hard to debug. IMO, Ant+Ivy is a lowest common denominator type of thing. Most people coming from the Java world know Ant, and it is mostly self-descriptive. Ivy is what SBT and Gradle use under the hood for dependency management and isn't too difficult to use on its own (there is some boilerplate involved). I think the main thing is we pick something all the devs are comfortable with. That said, Gradle does looks pretty nice
        Hide
        Joe Stein added a comment - - edited

        David Arthur Can you patch the README file also, I was following along with the path when reading it but got stuck with what to-do after I apply it. I want to see how it works in parrelel with the 0.8.0 sbt based release maybe something in 0.8.1 with some gradle too so we can decide for 0.9 ...

        Show
        Joe Stein added a comment - - edited David Arthur Can you patch the README file also, I was following along with the path when reading it but got stuck with what to-do after I apply it. I want to see how it works in parrelel with the 0.8.0 sbt based release maybe something in 0.8.1 with some gradle too so we can decide for 0.9 ...
        Hide
        Neha Narkhede added a comment -

        David Arthur +1 on investigating Gradle

        Show
        Neha Narkhede added a comment - David Arthur +1 on investigating Gradle
        Hide
        Sriram Subramanian added a comment -

        I have done some work on both maven and gradle. I found gradle to be really easy to setup and have my project up and running. I would vote for gradle from my experience.

        Show
        Sriram Subramanian added a comment - I have done some work on both maven and gradle. I found gradle to be really easy to setup and have my project up and running. I would vote for gradle from my experience.
        Hide
        Jay Kreps added a comment -

        We ended up doing gradle.

        Show
        Jay Kreps added a comment - We ended up doing gradle.
        Jay Kreps made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12775778 ] Apache Kafka Workflow [ 13050911 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13050911 ] no-reopen-closed, patch-avail [ 13054475 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        346d 9h 22m 1 Jay Kreps 20/Mar/14 22:05

          People

          • Assignee:
            Unassigned
            Reporter:
            David Arthur
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 168h
              168h
              Remaining:
              Remaining Estimate - 168h
              168h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development