Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Issue Links

        Activity

        Hide
        liorze Lior Zeno added a comment -

        This includes pom modification, documentation update, making sure that Jenkins is configured correctly with the right environment and jvm version, and finally, making sure that the project build succeeds and that all tests pass.

        Show
        liorze Lior Zeno added a comment - This includes pom modification, documentation update, making sure that Jenkins is configured correctly with the right environment and jvm version, and finally, making sure that the project build succeeds and that all tests pass.
        Hide
        mpercy Mike Percy added a comment -

        Hey Lior are you planning on submitting a patch for this? Just wondering because mention of it came up in FLUME-1899. /cc Hari Shreedharan

        Show
        mpercy Mike Percy added a comment - Hey Lior are you planning on submitting a patch for this? Just wondering because mention of it came up in FLUME-1899 . /cc Hari Shreedharan
        Hide
        liorze Lior Zeno added a comment -

        Yes, I'll submit a patch for this today. I assigned this to myself in order to make it clear that I'm working on this.

        Show
        liorze Lior Zeno added a comment - Yes, I'll submit a patch for this today. I assigned this to myself in order to make it clear that I'm working on this.
        Hide
        liorze Lior Zeno added a comment -

        I attached a patch. We still need to modify Jenkins and the wiki page, e.g. the Development Environment page in the Developer section. https://cwiki.apache.org/confluence/display/FLUME/Development+Environment

        All tests passed, except for kafka source, elasticsearch and solr, but we have separate jira issues for these:
        FLUME-2936, FLUME-2921 and FLUME-2919 respectively.
        I tested on openjdk 1.7.0_95 64bit.

        Finally, I took the opportunity in fixed a few warnings that came up during the build by maven.

        Show
        liorze Lior Zeno added a comment - I attached a patch. We still need to modify Jenkins and the wiki page, e.g. the Development Environment page in the Developer section. https://cwiki.apache.org/confluence/display/FLUME/Development+Environment All tests passed, except for kafka source, elasticsearch and solr, but we have separate jira issues for these: FLUME-2936 , FLUME-2921 and FLUME-2919 respectively. I tested on openjdk 1.7.0_95 64bit. Finally, I took the opportunity in fixed a few warnings that came up during the build by maven.
        Hide
        mpercy Mike Percy added a comment -

        Lior Zeno, thanks for the patch. There are a few problems with it:

        • It deletes the README.
        • Makes unrelated code changes (let's just keep this a pure build patch, plus any docs you want to update).
        • You didn't explain why you made changes involving removing components from the builds, please let me know so I can understand when reviewing.
        Show
        mpercy Mike Percy added a comment - Lior Zeno , thanks for the patch. There are a few problems with it: It deletes the README. Makes unrelated code changes (let's just keep this a pure build patch, plus any docs you want to update). You didn't explain why you made changes involving removing components from the builds, please let me know so I can understand when reviewing.
        Hide
        liorze Lior Zeno added a comment -

        I'll be much more verbose regarding the changes I've made in this patch.
        All the changes I've made are related to the build process and to the documentation.

        1. Renamed README to README.textile in order to allow github to properly render the headings, i.e. instead of rendering h1 it will actually render a heading. In addition, I changed the requirement to JDK 1.7 instead of JDK 1.6.
        2. FlumeUserGuide.rst: I changed the system requirements to Java 1.7 instead of 1.6 with a recommendation on 1.7. I did not add a recommendation for 1.8, since I do not see a reason for that. More over, I removed the note on the TailDirSource saying it requires 1.7, since it is the minimum required for the project.
        3. flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml: This pom file had two identical dependencies, therefore I removed one of them (httpclient). This was generating a warning in the beginning of the build.
        4. KafkaLocal.java: The tests threw a MethodNotFoundException, since there isn't a constructor for KafkaConfig with a java.util.properties parameter. Instead, I used the static method KafkaConfig.fromProps.
        5. Main pom.xml: I changed the sourceJavaVersion and targetJavaVersion to 1.7. In addition, I removed the test dependency hadoop-minicluster since it is already required as a "normal" dependency. Maven was complaining about that as well.

        Let me know if you want to split these changes into multiple small issues, or we can deal with them in this issue since it's about making Flume build properly under 1.7.

        Show
        liorze Lior Zeno added a comment - I'll be much more verbose regarding the changes I've made in this patch. All the changes I've made are related to the build process and to the documentation. Renamed README to README.textile in order to allow github to properly render the headings, i.e. instead of rendering h1 it will actually render a heading. In addition, I changed the requirement to JDK 1.7 instead of JDK 1.6. FlumeUserGuide.rst: I changed the system requirements to Java 1.7 instead of 1.6 with a recommendation on 1.7. I did not add a recommendation for 1.8, since I do not see a reason for that. More over, I removed the note on the TailDirSource saying it requires 1.7, since it is the minimum required for the project. flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml: This pom file had two identical dependencies, therefore I removed one of them (httpclient). This was generating a warning in the beginning of the build. KafkaLocal.java: The tests threw a MethodNotFoundException, since there isn't a constructor for KafkaConfig with a java.util.properties parameter. Instead, I used the static method KafkaConfig.fromProps. Main pom.xml: I changed the sourceJavaVersion and targetJavaVersion to 1.7. In addition, I removed the test dependency hadoop-minicluster since it is already required as a "normal" dependency. Maven was complaining about that as well. Let me know if you want to split these changes into multiple small issues, or we can deal with them in this issue since it's about making Flume build properly under 1.7.
        Hide
        mpercy Mike Percy added a comment -

        +1. Thanks for the explanation! I am about to commit this.

        Show
        mpercy Mike Percy added a comment - +1. Thanks for the explanation! I am about to commit this.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2f731f0aaeb490ab72400d8c83074daac840ac44 in flume's branch refs/heads/trunk from Mike Percy
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=2f731f0 ]

        FLUME-2935. Bump java target version to 1.7

        • Renamed README to README.textile in order to allow GitHub to properly
          render the headings. Also changed the min. requirement to JDK 1.7
        • pom.xml:
        • Changed the sourceJavaVersion and targetJavaVersion to 1.7
        • Removed the test dependency hadoop-minicluster since it is already
          required as a "normal" dependency. This was causing Maven warnings
        • flume-ng-elasticsearch-sink/pom.xml: pom file had two identical
          dependencies, causing Maven warnings
        • FlumeUserGuide.rst: changed the system requirements to Java 1.7
        • KafkaLocal.java: The tests threw a MethodNotFoundException, since
          there isn't a constructor for KafkaConfig with a java.util.properties
          parameter. Instead, I used the static method KafkaConfig.fromProps.

        (Lior Zeno via Mike Percy)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2f731f0aaeb490ab72400d8c83074daac840ac44 in flume's branch refs/heads/trunk from Mike Percy [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=2f731f0 ] FLUME-2935 . Bump java target version to 1.7 Renamed README to README.textile in order to allow GitHub to properly render the headings. Also changed the min. requirement to JDK 1.7 pom.xml: Changed the sourceJavaVersion and targetJavaVersion to 1.7 Removed the test dependency hadoop-minicluster since it is already required as a "normal" dependency. This was causing Maven warnings flume-ng-elasticsearch-sink/pom.xml: pom file had two identical dependencies, causing Maven warnings FlumeUserGuide.rst: changed the system requirements to Java 1.7 KafkaLocal.java: The tests threw a MethodNotFoundException, since there isn't a constructor for KafkaConfig with a java.util.properties parameter. Instead, I used the static method KafkaConfig.fromProps. (Lior Zeno via Mike Percy)
        Hide
        mpercy Mike Percy added a comment -

        Committed to trunk. Thanks for the patch, Lior!

        Show
        mpercy Mike Percy added a comment - Committed to trunk. Thanks for the patch, Lior!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Flume-trunk-hbase-1 #166 (See https://builds.apache.org/job/Flume-trunk-hbase-1/166/)
        FLUME-2935. Bump java target version to 1.7 (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=2f731f0aaeb490ab72400d8c83074daac840ac44)

        • flume-ng-doc/sphinx/FlumeUserGuide.rst
        • README
        • flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml
        • flume-ng-sinks/flume-ng-kafka-sink/src/test/java/org/apache/flume/sink/kafka/util/KafkaLocal.java
        • pom.xml
        • README.textile
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Flume-trunk-hbase-1 #166 (See https://builds.apache.org/job/Flume-trunk-hbase-1/166/ ) FLUME-2935 . Bump java target version to 1.7 (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=2f731f0aaeb490ab72400d8c83074daac840ac44 ) flume-ng-doc/sphinx/FlumeUserGuide.rst README flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml flume-ng-sinks/flume-ng-kafka-sink/src/test/java/org/apache/flume/sink/kafka/util/KafkaLocal.java pom.xml README.textile
        Hide
        liorze Lior Zeno added a comment -

        Thanks Mike!

        There are a few issues:

        1. Something went wrong with the patch. For instance, the README.textile file seems incorrect. Can you please take a look at it? Maybe there are other files that were merged incorrectly.
        1. The build fails due to rat check. Can you please upload rat's report? I don't really understand what went wrong here.
        Show
        liorze Lior Zeno added a comment - Thanks Mike! There are a few issues: Something went wrong with the patch. For instance, the README.textile file seems incorrect. Can you please take a look at it? Maybe there are other files that were merged incorrectly. The build fails due to rat check. Can you please upload rat's report? I don't really understand what went wrong here.
        Hide
        mpercy Mike Percy added a comment -

        Ah, yeah. I routinely skip rat when I build. You can usually find the RAT files in target/rat.txt and it turns out that the README was missing an ASL 2.0 license header.

        In order to unbreak the build, I just moved the README from .textile to .md (I couldn't figure out a .textile comment syntax that GitHub supports) and added a license comment to the top of the file. Trunk should be OK now.

        Show
        mpercy Mike Percy added a comment - Ah, yeah. I routinely skip rat when I build. You can usually find the RAT files in target/rat.txt and it turns out that the README was missing an ASL 2.0 license header. In order to unbreak the build, I just moved the README from .textile to .md (I couldn't figure out a .textile comment syntax that GitHub supports) and added a license comment to the top of the file. Trunk should be OK now.
        Hide
        liorze Lior Zeno added a comment -

        Thanks!
        It seems like we have a few issues with the tests:

        1. TestThriftLegacySource did not run for some reason (I can't reproduce this on my local machine).
        2. TestKafkaSource did not run - we have a jira for that. (FLUME-2936)
        3. The IT failed since flume-ng-distribution was skipped and the tarball was not generated. We should run mvn package on the dist project before running the IT tests.
        Show
        liorze Lior Zeno added a comment - Thanks! It seems like we have a few issues with the tests: TestThriftLegacySource did not run for some reason (I can't reproduce this on my local machine). TestKafkaSource did not run - we have a jira for that. ( FLUME-2936 ) The IT failed since flume-ng-distribution was skipped and the tarball was not generated. We should run mvn package on the dist project before running the IT tests.

          People

          • Assignee:
            liorze Lior Zeno
            Reporter:
            liorze Lior Zeno
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development