Kafka
  1. Kafka
  2. KAFKA-1054

Eliminate Compilation Warnings for 0.8 Final Release

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:

      Description

      Currently we have a total number of 38 warnings for source code compilation of 0.8.

      1) 3 from "Unchecked type pattern"
      2) 6 from "Unchecked conversion"
      3) 29 from "Deprecated Hadoop API functions"

      It's better we finish these before the final release of 0.8

      1. KAFKA-1054.patch
        5 kB
        Jonathan Riehl
      2. KAFKA-1054_Mar_10_2015.patch
        6 kB
        Blake Smith
      3. KAFKA-1054-20150426-V1.patch
        8 kB
        Ismael Juma
      4. KAFKA-1054-20150426-V2.patch
        7 kB
        Ismael Juma

        Activity

        Hide
        Guozhang Wang added a comment -

        I am not sure what to do with 3) but:

        For 1) it is mainly because of http://stackoverflow.com/questions/10298926/unchecked-type-pattern-warning-in-scala

        For 2) it is mainly because of missing type declarations.

        Show
        Guozhang Wang added a comment - I am not sure what to do with 3) but: For 1) it is mainly because of http://stackoverflow.com/questions/10298926/unchecked-type-pattern-warning-in-scala For 2) it is mainly because of missing type declarations.
        Hide
        Guozhang Wang added a comment -

        After KAFKA-1046 we still have all those warnings.

        Show
        Guozhang Wang added a comment - After KAFKA-1046 we still have all those warnings.
        Hide
        Jun Rao added a comment -

        This potentially needs some code refactoring. So, moving to 0.8.1.

        Show
        Jun Rao added a comment - This potentially needs some code refactoring. So, moving to 0.8.1.
        Hide
        Guozhang Wang added a comment -

        Moving to 0.9.

        Show
        Guozhang Wang added a comment - Moving to 0.9.
        Hide
        Jonathan Riehl added a comment - - edited

        Created reviewboard https://reviews.apache.org/r/25461/diff/
        against branch origin/trunk

        Not complete, but suppresses the majority of the Scala warnings.

        Show
        Jonathan Riehl added a comment - - edited Created reviewboard https://reviews.apache.org/r/25461/diff/ against branch origin/trunk Not complete, but suppresses the majority of the Scala warnings.
        Hide
        Neha Narkhede added a comment -

        Jonathan Riehl, patch looks great. Could you also address the remaining 2 warnings so we can close the JIRA?

        /Users/nnarkhed/Projects/kafka-git-idea/core/src/main/scala/kafka/network/SocketServer.scala:346: Visited SCOPE_EXIT before visiting corresponding SCOPE_ENTER. SI-6049
              try {
              ^
        there were 2 feature warning(s); re-run with -feature for details
        two warnings found
        :contrib:hadoop-consumer:compileJava
        Note: Some input files use or override a deprecated API.
        Note: Recompile with -Xlint:deprecation for details.
        Note: /Users/nnarkhed/Projects/kafka-git-idea/contrib/hadoop-consumer/src/main/java/kafka/etl/impl/DataGenerator.java uses unchecked or unsafe operations.
        
        Show
        Neha Narkhede added a comment - Jonathan Riehl , patch looks great. Could you also address the remaining 2 warnings so we can close the JIRA? /Users/nnarkhed/Projects/kafka-git-idea/core/src/main/scala/kafka/network/SocketServer.scala:346: Visited SCOPE_EXIT before visiting corresponding SCOPE_ENTER. SI-6049 try { ^ there were 2 feature warning(s); re-run with -feature for details two warnings found :contrib:hadoop-consumer:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: /Users/nnarkhed/Projects/kafka-git-idea/contrib/hadoop-consumer/src/main/java/kafka/etl/impl/DataGenerator.java uses unchecked or unsafe operations.
        Hide
        Blake Smith added a comment - - edited

        I attached an updated patch that's rebased on the current trunk, and squashed 2 more compiler warnings.

        Also: there are some surpressed scala "feature" warnings (looks like: postfix operators and implicit conversions). It looks like these warnings were added in 2.10 as part of SIP-18. I'm not sure what importing these language "features" will do in Scala 2.9 given that they were considered standard scala, but are they worth addressing?

        Feature warnings below:

        /Users/blake/src/kafka/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala:978: postfix operator contains should be enabled
        by making the implicit value scala.language.postfixOps visible.
        This can be achieved by adding the import clause 'import scala.language.postfixOps'
        or by setting the compiler option -language:postfixOps.
        See the Scala docs for value scala.language.postfixOps for a discussion
        why the feature should be explicitly enabled.
              val addedTopics = updatedTopics filterNot (wildcardTopics contains)
                                                                        ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala:988: postfix operator contains should be enabled
        by making the implicit value scala.language.postfixOps visible.
              val deletedTopics = wildcardTopics filterNot (updatedTopics contains)
                                                                          ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:23: implicit conversion method scalaMessageSetToJavaMessageSet should be enabled
        by making the implicit value scala.language.implicitConversions visible.
        This can be achieved by adding the import clause 'import scala.language.implicitConversions'
        or by setting the compiler option -language:implicitConversions.
        See the Scala docs for value scala.language.implicitConversions for a discussion
        why the feature should be explicitly enabled.
          implicit def scalaMessageSetToJavaMessageSet(messageSet: kafka.message.ByteBufferMessageSet):
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:28: implicit conversion method toJavaFetchResponse should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaFetchResponse(response: kafka.api.FetchResponse): kafka.javaapi.FetchResponse =
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:31: implicit conversion method toJavaTopicMetadataResponse should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaTopicMetadataResponse(response: kafka.api.TopicMetadataResponse): kafka.javaapi.TopicMetadataResponse =
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:34: implicit conversion method toJavaOffsetResponse should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaOffsetResponse(response: kafka.api.OffsetResponse): kafka.javaapi.OffsetResponse =
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:37: implicit conversion method toJavaOffsetFetchResponse should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaOffsetFetchResponse(response: kafka.api.OffsetFetchResponse): kafka.javaapi.OffsetFetchResponse =
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:40: implicit conversion method toJavaOffsetCommitResponse should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaOffsetCommitResponse(response: kafka.api.OffsetCommitResponse): kafka.javaapi.OffsetCommitResponse =
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:43: implicit conversion method optionToJavaRef should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def optionToJavaRef[T](opt: Option[T]): T = {
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:51: implicit conversion method javaListToScalaBuffer should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def javaListToScalaBuffer[A](l: java.util.List[A]) = {
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/TopicMetadata.scala:23: implicit conversion method toJavaTopicMetadataList should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toJavaTopicMetadataList(topicMetadataSeq: Seq[kafka.api.TopicMetadata]):
                       ^
        /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/TopicMetadata.scala:29: implicit conversion method toPartitionMetadataList should be enabled
        by making the implicit value scala.language.implicitConversions visible.
          implicit def toPartitionMetadataList(partitionMetadataSeq: Seq[kafka.api.PartitionMetadata]):
                       ^
        12 warnings found
        
        Show
        Blake Smith added a comment - - edited I attached an updated patch that's rebased on the current trunk, and squashed 2 more compiler warnings. Also: there are some surpressed scala "feature" warnings (looks like: postfix operators and implicit conversions). It looks like these warnings were added in 2.10 as part of SIP-18 . I'm not sure what importing these language "features" will do in Scala 2.9 given that they were considered standard scala, but are they worth addressing? Feature warnings below: /Users/blake/src/kafka/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala:978: postfix operator contains should be enabled by making the implicit value scala.language.postfixOps visible. This can be achieved by adding the import clause ' import scala.language.postfixOps' or by setting the compiler option -language:postfixOps. See the Scala docs for value scala.language.postfixOps for a discussion why the feature should be explicitly enabled. val addedTopics = updatedTopics filterNot (wildcardTopics contains) ^ /Users/blake/src/kafka/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala:988: postfix operator contains should be enabled by making the implicit value scala.language.postfixOps visible. val deletedTopics = wildcardTopics filterNot (updatedTopics contains) ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:23: implicit conversion method scalaMessageSetToJavaMessageSet should be enabled by making the implicit value scala.language.implicitConversions visible. This can be achieved by adding the import clause ' import scala.language.implicitConversions' or by setting the compiler option -language:implicitConversions. See the Scala docs for value scala.language.implicitConversions for a discussion why the feature should be explicitly enabled. implicit def scalaMessageSetToJavaMessageSet(messageSet: kafka.message.ByteBufferMessageSet): ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:28: implicit conversion method toJavaFetchResponse should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaFetchResponse(response: kafka.api.FetchResponse): kafka.javaapi.FetchResponse = ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:31: implicit conversion method toJavaTopicMetadataResponse should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaTopicMetadataResponse(response: kafka.api.TopicMetadataResponse): kafka.javaapi.TopicMetadataResponse = ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:34: implicit conversion method toJavaOffsetResponse should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaOffsetResponse(response: kafka.api.OffsetResponse): kafka.javaapi.OffsetResponse = ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:37: implicit conversion method toJavaOffsetFetchResponse should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaOffsetFetchResponse(response: kafka.api.OffsetFetchResponse): kafka.javaapi.OffsetFetchResponse = ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:40: implicit conversion method toJavaOffsetCommitResponse should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaOffsetCommitResponse(response: kafka.api.OffsetCommitResponse): kafka.javaapi.OffsetCommitResponse = ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:43: implicit conversion method optionToJavaRef should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def optionToJavaRef[T](opt: Option[T]): T = { ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/Implicits.scala:51: implicit conversion method javaListToScalaBuffer should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def javaListToScalaBuffer[A](l: java.util.List[A]) = { ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/TopicMetadata.scala:23: implicit conversion method toJavaTopicMetadataList should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toJavaTopicMetadataList(topicMetadataSeq: Seq[kafka.api.TopicMetadata]): ^ /Users/blake/src/kafka/core/src/main/scala/kafka/javaapi/TopicMetadata.scala:29: implicit conversion method toPartitionMetadataList should be enabled by making the implicit value scala.language.implicitConversions visible. implicit def toPartitionMetadataList(partitionMetadataSeq: Seq[kafka.api.PartitionMetadata]): ^ 12 warnings found
        Hide
        Blake Smith added a comment -

        To follow up on this, it looks like we can't kill the feature warnings until we stop supporting scala 2.9. The language implicit "feature flags" were brought in with scala 2.10 and won't compile with 2.9.

        Show
        Blake Smith added a comment - To follow up on this, it looks like we can't kill the feature warnings until we stop supporting scala 2.9. The language implicit "feature flags" were brought in with scala 2.10 and won't compile with 2.9.
        Hide
        Jay Kreps added a comment -

        Neha Narkhede I think this is waiting on you.

        Show
        Jay Kreps added a comment - Neha Narkhede I think this is waiting on you.
        Hide
        Ismael Juma added a comment -

        Note that there are also some deprecation warnings when kafka is compiled with Scala 2.11 (mostly usage of `Pair`). Blake Smith, are you interested in updating your patch to include these as well? If not, I am happy to do it.

        Show
        Ismael Juma added a comment - Note that there are also some deprecation warnings when kafka is compiled with Scala 2.11 (mostly usage of `Pair`). Blake Smith , are you interested in updating your patch to include these as well? If not, I am happy to do it.
        Hide
        Neha Narkhede added a comment -

        Sorry, I missed checking back on this. Blake Smith your changes look good. Is anyone planning on addressing the changes Ismael Juma suggested? I'm wondering if I can push this just yet.

        Show
        Neha Narkhede added a comment - Sorry, I missed checking back on this. Blake Smith your changes look good. Is anyone planning on addressing the changes Ismael Juma suggested? I'm wondering if I can push this just yet.
        Hide
        Ismael Juma added a comment -

        This is a rebased patch with an additional commit for fixing Scala 2.11 warnings.

        Show
        Ismael Juma added a comment - This is a rebased patch with an additional commit for fixing Scala 2.11 warnings.
        Hide
        Ismael Juma added a comment -

        This is the same as `KAFKA-1054-20150426-V1.patch`, but commit "Removed code with no discernable effect that was causing warnings" has been removed.

        I am not sure if that is a safe change as the calls that have been removed may potentially trigger side-effects (e.g. `newMeter("UncleanLeaderElectionsPerSec", "elections", TimeUnit.SECONDS)`).

        Show
        Ismael Juma added a comment - This is the same as ` KAFKA-1054 -20150426-V1.patch`, but commit "Removed code with no discernable effect that was causing warnings" has been removed. I am not sure if that is a safe change as the calls that have been removed may potentially trigger side-effects (e.g. `newMeter("UncleanLeaderElectionsPerSec", "elections", TimeUnit.SECONDS)`).
        Hide
        ASF GitHub Bot added a comment -

        GitHub user ijuma opened a pull request:

        https://github.com/apache/kafka/pull/57

        KAFKA-1054; Eliminate Scala Compilation Warnings

        Changes:

        • Suppressed compiler warnings about type erasure in matching via unboxing
          by Jon Riehl.
        • Suppressed warning caused by slight difference in input function type
          by John Riehl.
        • Fix compiler warnings: ServerShutdownTest, DelayedJoinGroup function
          signature by Blake Smith.
        • Fix Scala 2.11 warnings. `Pair` has been deprecated, `try` without
          `catch` and `finally` is useless and initialisation order fix by Ismael
          Juma.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/ijuma/kafka kafka-1054-squashed

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/kafka/pull/57.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #57


        commit 0a5501c84208ea2cb734cc2c2e30735503716f32
        Author: Jon Riehl <jriehl@spaceship.com>
        Date: 2014-09-05T23:25:32Z

        KAFKA-1054; Eliminate Scala Compilation Warnings

        Changes:

        • Suppressed compiler warnings about type erasure in matching via unboxing
          by Jon Riehl.
        • Suppressed warning caused by slight difference in input function type
          by John Riehl.
        • Fix compiler warnings: ServerShutdownTest, DelayedJoinGroup function
          signature by Blake Smith.
        • Fix Scala 2.11 warnings. `Pair` has been deprecated, `try` without
          `catch` and `finally` is useless and initialisation order fix by Ismael
          Juma.

        Show
        ASF GitHub Bot added a comment - GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/57 KAFKA-1054 ; Eliminate Scala Compilation Warnings Changes: Suppressed compiler warnings about type erasure in matching via unboxing by Jon Riehl. Suppressed warning caused by slight difference in input function type by John Riehl. Fix compiler warnings: ServerShutdownTest, DelayedJoinGroup function signature by Blake Smith. Fix Scala 2.11 warnings. `Pair` has been deprecated, `try` without `catch` and `finally` is useless and initialisation order fix by Ismael Juma. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-1054-squashed Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/57.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #57 commit 0a5501c84208ea2cb734cc2c2e30735503716f32 Author: Jon Riehl <jriehl@spaceship.com> Date: 2014-09-05T23:25:32Z KAFKA-1054 ; Eliminate Scala Compilation Warnings Changes: Suppressed compiler warnings about type erasure in matching via unboxing by Jon Riehl. Suppressed warning caused by slight difference in input function type by John Riehl. Fix compiler warnings: ServerShutdownTest, DelayedJoinGroup function signature by Blake Smith. Fix Scala 2.11 warnings. `Pair` has been deprecated, `try` without `catch` and `finally` is useless and initialisation order fix by Ismael Juma.
        Hide
        Ismael Juma added a comment -

        A PR with all the commits squashed:

        https://github.com/apache/kafka/pull/57

        Show
        Ismael Juma added a comment - A PR with all the commits squashed: https://github.com/apache/kafka/pull/57
        Hide
        Neha Narkhede added a comment -

        Thanks. Merged the github PR.

        Show
        Neha Narkhede added a comment - Thanks. Merged the github PR.
        Hide
        ASF GitHub Bot added a comment -

        Github user ijuma closed the pull request at:

        https://github.com/apache/kafka/pull/57

        Show
        ASF GitHub Bot added a comment - Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/57

          People

          • Assignee:
            Ismael Juma
            Reporter:
            Guozhang Wang
            Reviewer:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development