Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: build
    • Labels:
      None

      Description

      Bump up Apache Phoenix version to 4.5.0

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          aH, great! Thanks for finishing this, YoungWoo Kim

          Show
          cos Konstantin Boudnik added a comment - aH, great! Thanks for finishing this, YoungWoo Kim
          Hide
          warwithin YoungWoo Kim added a comment -

          Committed. Thanks Jonathan Kelly!

          Show
          warwithin YoungWoo Kim added a comment - Committed. Thanks Jonathan Kelly !
          Hide
          warwithin YoungWoo Kim added a comment -

          LGTM! Running tests now...

          Show
          warwithin YoungWoo Kim added a comment - LGTM! Running tests now...
          Hide
          cos Konstantin Boudnik added a comment -

          Looks like this becomes critical now because the Phoenix build is broken after Spark 1.5.1 upgrade.

          Show
          cos Konstantin Boudnik added a comment - Looks like this becomes critical now because the Phoenix build is broken after Spark 1.5.1 upgrade.
          Hide
          jonathak Jonathan Kelly added a comment -

          I was able to build a Phoenix 4.6.0 RPM with Spark 1.5.1, so I created a pull request. I have not actually tried using Phoenix yet though, as I don't yet have any experience with it or really much experience with HBase either. Andrew Purtell or YoungWoo Kim, since you are listed as the Phoenix maintainers, are you able to test this?

          Show
          jonathak Jonathan Kelly added a comment - I was able to build a Phoenix 4.6.0 RPM with Spark 1.5.1, so I created a pull request. I have not actually tried using Phoenix yet though, as I don't yet have any experience with it or really much experience with HBase either. Andrew Purtell or YoungWoo Kim , since you are listed as the Phoenix maintainers, are you able to test this?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user ejono opened a pull request:

          https://github.com/apache/bigtop/pull/48

          BIGTOP-1942. Upgrade Phoenix to 4.6.0

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/ejono/bigtop BIGTOP-1942

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/48.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #48


          commit 56b4dbc2d41a1e1987e6761ef94f2fadfc6565b7
          Author: Jonathan Kelly <jonathak@amazon.com>
          Date: 2015-11-02T22:09:45Z

          BIGTOP-1942. Upgrade Phoenix to 4.6.0


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user ejono opened a pull request: https://github.com/apache/bigtop/pull/48 BIGTOP-1942 . Upgrade Phoenix to 4.6.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/ejono/bigtop BIGTOP-1942 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/48.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #48 commit 56b4dbc2d41a1e1987e6761ef94f2fadfc6565b7 Author: Jonathan Kelly <jonathak@amazon.com> Date: 2015-11-02T22:09:45Z BIGTOP-1942 . Upgrade Phoenix to 4.6.0
          Hide
          warwithin YoungWoo Kim added a comment -

          Jonathan Kelly Sounds reasonable! I've just updated the summary.

          Show
          warwithin YoungWoo Kim added a comment - Jonathan Kelly Sounds reasonable! I've just updated the summary.
          Hide
          jonathak Jonathan Kelly added a comment -

          Phoenix 4.6.0 was just released, so we should investigate upgrading to it rather than to 4.5.3, which has not been released yet. I have not tried building Phoenix 4.6 with Spark 1.5 yet, but I think it supposedly will work.

          Show
          jonathak Jonathan Kelly added a comment - Phoenix 4.6.0 was just released, so we should investigate upgrading to it rather than to 4.5.3, which has not been released yet. I have not tried building Phoenix 4.6 with Spark 1.5 yet, but I think it supposedly will work.
          Hide
          apurtell Andrew Purtell added a comment -

          Updated the issue summary to not specify a minor version for Phoenix 4.5.x. We can track the releases in that line and pick the latest, if it works, or adjust this issue further to pick up a release in 4.6.x instead.

          Show
          apurtell Andrew Purtell added a comment - Updated the issue summary to not specify a minor version for Phoenix 4.5.x. We can track the releases in that line and pick the latest, if it works, or adjust this issue further to pick up a release in 4.6.x instead.
          Hide
          apurtell Andrew Purtell added a comment -

          I found a related issue PHOENIX-2287, which suggests Phoenix 4.5.3 or 4.6.x may work with Spark 1.5+

          Show
          apurtell Andrew Purtell added a comment - I found a related issue PHOENIX-2287 , which suggests Phoenix 4.5.3 or 4.6.x may work with Spark 1.5+
          Hide
          apurtell Andrew Purtell added a comment -

          Are we sure that upgrading to Phoenix 4.5.0 will allow us to upgrade to Spark 1.5?
          ...

          error: object UTF8String is not a member of package org.apache.spark.sql.types
          

          Ugh, that's unfortunate. I haven't tried this myself but by appearance Phoenix is a blocker to moving up to Spark 1.5, because Spark changed the SparkSQL APIs in that version and Phoenix needs tweaking in response.

          Show
          apurtell Andrew Purtell added a comment - Are we sure that upgrading to Phoenix 4.5.0 will allow us to upgrade to Spark 1.5? ... error: object UTF8String is not a member of package org.apache.spark.sql.types Ugh, that's unfortunate. I haven't tried this myself but by appearance Phoenix is a blocker to moving up to Spark 1.5, because Spark changed the SparkSQL APIs in that version and Phoenix needs tweaking in response.
          Hide
          jonathak Jonathan Kelly added a comment -

          Are we sure that upgrading to Phoenix 4.5.0 will allow us to upgrade to Spark 1.5? I get the following when I try to compile Phoenix 4.5.0 in Bigtop with Spark 1.5.1:

          [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRDD.scala:140: error: type mismatch;
          [ERROR]  found   : Unit
          [ERROR]  required: org.apache.spark.sql.types.DataType
          [ERROR]     case t if t.isInstanceOf[PDecimal] => DecimalType(None)
          [ERROR]                                                      ^
          [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRDD.scala:155: error: type mismatch;
          [ERROR]  found   : Unit
          [ERROR]  required: org.apache.spark.sql.types.DataType
          [ERROR]     case t if t.isInstanceOf[PDecimalArray] => ArrayType(DecimalType(None), containsNull = true)
          [ERROR]                                                                     ^
          ...
          [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRelation.scala:22: error: object UTF8String is not a member of package org.apache.spark.sql.types
          [ERROR] import org.apache.spark.sql.types.{UTF8String, StructType}
          [ERROR]        ^
          [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRelation.scala:95: error: not found: type UTF8String
          [ERROR]     case stringValue: UTF8String => s"'${escapeStringConstant(stringValue.toString)}'"
          [ERROR]                       ^
          
          Show
          jonathak Jonathan Kelly added a comment - Are we sure that upgrading to Phoenix 4.5.0 will allow us to upgrade to Spark 1.5? I get the following when I try to compile Phoenix 4.5.0 in Bigtop with Spark 1.5.1: [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRDD.scala:140: error: type mismatch; [ERROR] found : Unit [ERROR] required: org.apache.spark.sql.types.DataType [ERROR] case t if t.isInstanceOf[PDecimal] => DecimalType(None) [ERROR] ^ [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRDD.scala:155: error: type mismatch; [ERROR] found : Unit [ERROR] required: org.apache.spark.sql.types.DataType [ERROR] case t if t.isInstanceOf[PDecimalArray] => ArrayType(DecimalType(None), containsNull = true ) [ERROR] ^ ... [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRelation.scala:22: error: object UTF8String is not a member of package org.apache.spark.sql.types [ERROR] import org.apache.spark.sql.types.{UTF8String, StructType} [ERROR] ^ [ERROR] /workspace/workspace/jonathak.bigtop.phoenix-rpm/build/phoenix/rpm/BUILD/phoenix-4.5.0-HBase-0.98-src/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRelation.scala:95: error: not found: type UTF8String [ERROR] case stringValue: UTF8String => s "'${escapeStringConstant(stringValue.toString)}'" [ERROR] ^

            People

            • Assignee:
              jonathak Jonathan Kelly
              Reporter:
              warwithin YoungWoo Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development