Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1574

Phoenix packaging requires build and classpath changes after PHOENIX-1455

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0
    • Component/s: None
    • Labels:
      None

      Description

      PHOENIX-1455 replaces org.xerial.snappy with org.iq80.snappy. There are two issues as initially reported on PHOENIX JIRAs.

      1. PHOENIX-1530: Bigtop shouldn't define 'snappy.version' for the Phoenix build, since there isn't agreement between the build scaffold and Phoenix as to which snappy implementation is wanted.
      2. PHOENIX-1531: We need a minor fix to how HBASE_CLASSPATH is set up. We don't want the "core" jar, we want the "client" jar.
      1. BIGTOP-1574.patch
        4 kB
        Andrew Purtell

        Issue Links

          Activity

          Hide
          apurtell Andrew Purtell added a comment -

          Pushed

          Show
          apurtell Andrew Purtell added a comment - Pushed
          Hide
          rvs Roman Shaposhnik added a comment -

          +1. Can you guys please commit this?

          Show
          rvs Roman Shaposhnik added a comment - +1. Can you guys please commit this?
          Hide
          jayunit100 jay vyas added a comment -

          Looks good to me ... looks like generally just cleaning up the phoenix packaging, and all the changes seem to be generic (i.e. will work the same in RPM as deb)

          Show
          jayunit100 jay vyas added a comment - Looks good to me ... looks like generally just cleaning up the phoenix packaging, and all the changes seem to be generic (i.e. will work the same in RPM as deb)
          Hide
          apurtell Andrew Purtell added a comment -

          PHOENIX-1532 was checked in, so once there's a release containing that the workaround in this patch can be dropped.

          Show
          apurtell Andrew Purtell added a comment - PHOENIX-1532 was checked in, so once there's a release containing that the workaround in this patch can be dropped.
          Hide
          apurtell Andrew Purtell added a comment -

          Changes:

          • Update to Phoenix latest 4.2.2
          • Don't set snappy.version
          • Don't specify hadoop.profile, it's not meaningful
          • Use 'hadoop.version' instead of 'hadoop-two.version'
          • Use 'hbase.version' instead of 'hbase-hadoop2.version'
          • Workaround for PHOENIX-1532: Copy phoenix-server jar into build directory
          • Add phoenix-server.jar to HBASE_CLASSPATH in phoenix.default instead of phoenix-core

          All changes to common files, checked with a deb build on Ubuntu.

          Show
          apurtell Andrew Purtell added a comment - Changes: Update to Phoenix latest 4.2.2 Don't set snappy.version Don't specify hadoop.profile, it's not meaningful Use 'hadoop.version' instead of 'hadoop-two.version' Use 'hbase.version' instead of 'hbase-hadoop2.version' Workaround for PHOENIX-1532 : Copy phoenix-server jar into build directory Add phoenix-server.jar to HBASE_CLASSPATH in phoenix.default instead of phoenix-core All changes to common files, checked with a deb build on Ubuntu.
          Hide
          apurtell Andrew Purtell added a comment -

          I have a patch and am testing it now locally with a deb build. Will post it when that finishes.

          Show
          apurtell Andrew Purtell added a comment - I have a patch and am testing it now locally with a deb build. Will post it when that finishes.
          Hide
          jayunit100 jay vyas added a comment - - edited

          hi folks, i just created a JIRA for puppetization of phoenix and auto testing it... would be cool to run it in the vagrant testers ! (We can discuss in BIGTOP-1575)

          Show
          jayunit100 jay vyas added a comment - - edited hi folks, i just created a JIRA for puppetization of phoenix and auto testing it... would be cool to run it in the vagrant testers ! (We can discuss in BIGTOP-1575 )
          Hide
          apurtell Andrew Purtell added a comment -

          It is meant to be -server for the classpath here, it's the integration for the Phoenix server side. The client side integration, using the -client jar, will be untouched.

          Show
          apurtell Andrew Purtell added a comment - It is meant to be -server for the classpath here, it's the integration for the Phoenix server side. The client side integration, using the -client jar, will be untouched.
          Hide
          apurtell Andrew Purtell added a comment -

          Please wait for a patch. It will be tested.

          Show
          apurtell Andrew Purtell added a comment - Please wait for a patch. It will be tested.
          Hide
          jchabot72 Jerry Chabot added a comment - - edited

          I've only tested with the core AND client. Not just the client. Will it work with just the client jar?

          Show
          jchabot72 Jerry Chabot added a comment - - edited I've only tested with the core AND client. Not just the client. Will it work with just the client jar?
          Hide
          apurtell Andrew Purtell added a comment -

          Yes you are right Jerry Chabot, that was a typo, I have fixed it. Patch coming shortly.

          Show
          apurtell Andrew Purtell added a comment - Yes you are right Jerry Chabot , that was a typo, I have fixed it. Patch coming shortly.
          Hide
          jchabot72 Jerry Chabot added a comment -

          This bug mentions using the "server" jar. I don't see that jar after installing the RPM. I configured the classpath with the phoenix-client.jar which appears to work.
          $ ls -ls /usr/lib/phoenix/
          total 34592
          0 drwxr-xr-x 2 root root 149 Dec 15 18:12 bin
          4 drwxr-xr-x 2 root root 4096 Dec 15 18:12 lib
          31812 rw-rr- 1 root root 32571642 Dec 15 17:19 phoenix-4.2.2-client.jar
          0 lrwxrwxrwx 1 root root 24 Dec 15 18:12 phoenix-client.jar -> phoenix-4.2.2-client.jar
          2776 rw-rr- 1 root root 2840795 Dec 15 17:18 phoenix-core-4.2.2.jar
          0 lrwxrwxrwx 1 root root 22 Dec 15 18:12 phoenix-core.jar -> phoenix-core-4.2.2.jar

          ls -ls /usr/lib/phoenix/lib/
          total 11056
          1120 rw-rr- 1 root root 1144670 Oct 23 19:45 antlr-3.5.jar
          256 rw-rr- 1 root root 259600 Oct 23 17:19 commons-codec-1.7.jar
          292 rw-rr- 1 root root 298829 Oct 23 16:51 commons-configuration-1.6.jar
          36 rw-rr- 1 root root 34827 Nov 5 21:20 commons-csv-1.0.jar
          184 rw-rr- 1 root root 185140 Oct 23 16:50 commons-io-2.4.jar
          280 rw-rr- 1 root root 284220 Oct 23 16:50 commons-lang-2.6.jar
          60 rw-rr- 1 root root 60686 Oct 23 16:48 commons-logging-1.1.1.jar
          1756 rw-rr- 1 root root 1795932 Oct 23 17:19 guava-12.0.1.jar
          0 lrwxrwxrwx 1 root root 38 Dec 15 18:12 hadoop-annotations.jar -> /usr/lib/hadoop/hadoop-annotations.jar
          0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hadoop-auth.jar -> /usr/lib/hadoop/hadoop-auth.jar
          0 lrwxrwxrwx 1 root root 33 Dec 15 18:12 hadoop-common.jar -> /usr/lib/hadoop/hadoop-common.jar
          0 lrwxrwxrwx 1 root root 36 Dec 15 18:12 hadoop-hdfs.jar -> /usr/lib/hadoop-hdfs/hadoop-hdfs.jar
          0 lrwxrwxrwx 1 root root 58 Dec 15 18:12 hadoop-mapreduce-client-core.jar -> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar
          0 lrwxrwxrwx 1 root root 40 Dec 15 18:12 hadoop-yarn-api.jar -> /usr/lib/hadoop-yarn/hadoop-yarn-api.jar
          0 lrwxrwxrwx 1 root root 43 Dec 15 18:12 hadoop-yarn-common.jar -> /usr/lib/hadoop-yarn/hadoop-yarn-common.jar
          0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hbase-client.jar -> /usr/lib/hbase/hbase-client.jar
          0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hbase-common.jar -> /usr/lib/hbase/hbase-common.jar
          0 lrwxrwxrwx 1 root root 27 Dec 15 18:12 hbase-it.jar -> /usr/lib/hbase/hbase-it.jar
          0 lrwxrwxrwx 1 root root 33 Dec 15 18:12 hbase-protocol.jar -> /usr/lib/hbase/hbase-protocol.jar
          0 lrwxrwxrwx 1 root root 37 Dec 15 18:12 hbase-testing-util.jar -> /usr/lib/hbase/hbase-testing-util.jar
          32 rw-rr- 1 root root 31532 Oct 23 17:20 htrace-core-2.04.jar
          224 rw-rr- 1 root root 227500 Oct 23 16:51 jackson-core-asl-1.8.8.jar
          656 rw-rr- 1 root root 668564 Oct 23 16:51 jackson-mapper-asl-1.8.8.jar
          480 rw-rr- 1 root root 489884 Oct 23 16:50 log4j-1.2.17.jar
          1180 rw-rr- 1 root root 1206119 Oct 23 17:19 netty-3.6.6.Final.jar
          2776 rw-rr- 1 root root 2840795 Dec 15 17:18 phoenix-core-4.2.2.jar
          1008 rw-rr- 1 root root 1029838 Dec 15 17:18 phoenix-core-4.2.2-tests.jar
          36 rw-rr- 1 root root 33110 Dec 15 17:18 phoenix-flume-4.2.2.jar
          20 rw-rr- 1 root root 19479 Dec 15 17:18 phoenix-flume-4.2.2-tests.jar
          60 rw-rr- 1 root root 58633 Dec 15 17:18 phoenix-pig-4.2.2.jar
          36 rw-rr- 1 root root 33210 Dec 15 17:18 phoenix-pig-4.2.2-tests.jar
          524 rw-rr- 1 root root 533455 Oct 23 16:51 protobuf-java-2.5.0.jar
          28 rw-rr- 1 root root 25962 Oct 23 16:52 slf4j-api-1.6.4.jar
          12 rw-rr- 1 root root 8869 Oct 23 16:50 slf4j-log4j12-1.7.5.jar
          0 lrwxrwxrwx 1 root root 32 Dec 15 18:12 zookeeper.jar -> /usr/lib/zookeeper/zookeeper.jar

          Show
          jchabot72 Jerry Chabot added a comment - This bug mentions using the "server" jar. I don't see that jar after installing the RPM. I configured the classpath with the phoenix-client.jar which appears to work. $ ls -ls /usr/lib/phoenix/ total 34592 0 drwxr-xr-x 2 root root 149 Dec 15 18:12 bin 4 drwxr-xr-x 2 root root 4096 Dec 15 18:12 lib 31812 rw-r r - 1 root root 32571642 Dec 15 17:19 phoenix-4.2.2-client.jar 0 lrwxrwxrwx 1 root root 24 Dec 15 18:12 phoenix-client.jar -> phoenix-4.2.2-client.jar 2776 rw-r r - 1 root root 2840795 Dec 15 17:18 phoenix-core-4.2.2.jar 0 lrwxrwxrwx 1 root root 22 Dec 15 18:12 phoenix-core.jar -> phoenix-core-4.2.2.jar ls -ls /usr/lib/phoenix/lib/ total 11056 1120 rw-r r - 1 root root 1144670 Oct 23 19:45 antlr-3.5.jar 256 rw-r r - 1 root root 259600 Oct 23 17:19 commons-codec-1.7.jar 292 rw-r r - 1 root root 298829 Oct 23 16:51 commons-configuration-1.6.jar 36 rw-r r - 1 root root 34827 Nov 5 21:20 commons-csv-1.0.jar 184 rw-r r - 1 root root 185140 Oct 23 16:50 commons-io-2.4.jar 280 rw-r r - 1 root root 284220 Oct 23 16:50 commons-lang-2.6.jar 60 rw-r r - 1 root root 60686 Oct 23 16:48 commons-logging-1.1.1.jar 1756 rw-r r - 1 root root 1795932 Oct 23 17:19 guava-12.0.1.jar 0 lrwxrwxrwx 1 root root 38 Dec 15 18:12 hadoop-annotations.jar -> /usr/lib/hadoop/hadoop-annotations.jar 0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hadoop-auth.jar -> /usr/lib/hadoop/hadoop-auth.jar 0 lrwxrwxrwx 1 root root 33 Dec 15 18:12 hadoop-common.jar -> /usr/lib/hadoop/hadoop-common.jar 0 lrwxrwxrwx 1 root root 36 Dec 15 18:12 hadoop-hdfs.jar -> /usr/lib/hadoop-hdfs/hadoop-hdfs.jar 0 lrwxrwxrwx 1 root root 58 Dec 15 18:12 hadoop-mapreduce-client-core.jar -> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar 0 lrwxrwxrwx 1 root root 40 Dec 15 18:12 hadoop-yarn-api.jar -> /usr/lib/hadoop-yarn/hadoop-yarn-api.jar 0 lrwxrwxrwx 1 root root 43 Dec 15 18:12 hadoop-yarn-common.jar -> /usr/lib/hadoop-yarn/hadoop-yarn-common.jar 0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hbase-client.jar -> /usr/lib/hbase/hbase-client.jar 0 lrwxrwxrwx 1 root root 31 Dec 15 18:12 hbase-common.jar -> /usr/lib/hbase/hbase-common.jar 0 lrwxrwxrwx 1 root root 27 Dec 15 18:12 hbase-it.jar -> /usr/lib/hbase/hbase-it.jar 0 lrwxrwxrwx 1 root root 33 Dec 15 18:12 hbase-protocol.jar -> /usr/lib/hbase/hbase-protocol.jar 0 lrwxrwxrwx 1 root root 37 Dec 15 18:12 hbase-testing-util.jar -> /usr/lib/hbase/hbase-testing-util.jar 32 rw-r r - 1 root root 31532 Oct 23 17:20 htrace-core-2.04.jar 224 rw-r r - 1 root root 227500 Oct 23 16:51 jackson-core-asl-1.8.8.jar 656 rw-r r - 1 root root 668564 Oct 23 16:51 jackson-mapper-asl-1.8.8.jar 480 rw-r r - 1 root root 489884 Oct 23 16:50 log4j-1.2.17.jar 1180 rw-r r - 1 root root 1206119 Oct 23 17:19 netty-3.6.6.Final.jar 2776 rw-r r - 1 root root 2840795 Dec 15 17:18 phoenix-core-4.2.2.jar 1008 rw-r r - 1 root root 1029838 Dec 15 17:18 phoenix-core-4.2.2-tests.jar 36 rw-r r - 1 root root 33110 Dec 15 17:18 phoenix-flume-4.2.2.jar 20 rw-r r - 1 root root 19479 Dec 15 17:18 phoenix-flume-4.2.2-tests.jar 60 rw-r r - 1 root root 58633 Dec 15 17:18 phoenix-pig-4.2.2.jar 36 rw-r r - 1 root root 33210 Dec 15 17:18 phoenix-pig-4.2.2-tests.jar 524 rw-r r - 1 root root 533455 Oct 23 16:51 protobuf-java-2.5.0.jar 28 rw-r r - 1 root root 25962 Oct 23 16:52 slf4j-api-1.6.4.jar 12 rw-r r - 1 root root 8869 Oct 23 16:50 slf4j-log4j12-1.7.5.jar 0 lrwxrwxrwx 1 root root 32 Dec 15 18:12 zookeeper.jar -> /usr/lib/zookeeper/zookeeper.jar

            People

            • Assignee:
              apurtell Andrew Purtell
              Reporter:
              apurtell Andrew Purtell
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development