Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1448

Include zookeeper C library in bigtop

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: general
    • Labels:
      None

      Description

      It is nice to include to zookeeper C library in bigtop.

      how to do?

      1. add c build tool chain in bigtop

      2. build zookeeper c library as http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html

      3. package zookeeper c library

        Activity

        Hide
        rvs Roman Shaposhnik added a comment -

        I believe that Sean Mackrory had the C bindings packaged at some point. Sean, any chance you can share the packaging code?

        Show
        rvs Roman Shaposhnik added a comment - I believe that Sean Mackrory had the C bindings packaged at some point. Sean, any chance you can share the packaging code?
        Hide
        mackrorysd Sean Mackrory added a comment -

        Sure can. I thought I already had, but if not I will post a patch this week.

        Show
        mackrorysd Sean Mackrory added a comment - Sure can. I thought I already had, but if not I will post a patch this week.
        Hide
        mackrorysd Sean Mackrory added a comment -

        Attaching a patch. The RPM looks about right but I haven't tested the DEB in a while. This should allow you to run the single- and multi-threaded clients (cli_st and clit_mt, respectively) and the load tester (load_gen). It also allows you to include ZK headers (#include <zookeeper/zookeeper.h>) and link against the libraries (-lzookeeper). There are probably a few C things we need to add to bigtop toolchain (including cppunit-devel).

        One weird thing about this is that there are 2 tarballs in build/. There's zookeeper-$

        {VERSION}.tar.gz (which is what we've always consumed in install_zookeeper.sh) and zookeeper-${VERSION}

        -lib.tar.gz (which provides all the native bits). It works, but that's something kinda unique about this so I wanted to point it out.

        Show
        mackrorysd Sean Mackrory added a comment - Attaching a patch. The RPM looks about right but I haven't tested the DEB in a while. This should allow you to run the single- and multi-threaded clients (cli_st and clit_mt, respectively) and the load tester (load_gen). It also allows you to include ZK headers (#include <zookeeper/zookeeper.h>) and link against the libraries (-lzookeeper). There are probably a few C things we need to add to bigtop toolchain (including cppunit-devel). One weird thing about this is that there are 2 tarballs in build/. There's zookeeper-$ {VERSION}.tar.gz (which is what we've always consumed in install_zookeeper.sh) and zookeeper-${VERSION} -lib.tar.gz (which provides all the native bits). It works, but that's something kinda unique about this so I wanted to point it out.
        Hide
        rvs Roman Shaposhnik added a comment -

        Sean Mackrory cppunit-devel is already in the toolchain – don't worry about that. It would be really appreciated if you could provide any updates on the Debian side. Would love to see this committed to trunk ASAP.

        Show
        rvs Roman Shaposhnik added a comment - Sean Mackrory cppunit-devel is already in the toolchain – don't worry about that. It would be really appreciated if you could provide any updates on the Debian side. Would love to see this committed to trunk ASAP.
        Hide
        mackrorysd Sean Mackrory added a comment -

        So packages built on Debian are looking good too. I didn't make any substantial changes - I just rebased on top of the patch to add the zookeeper-rest package, and I removed my "TODO" note to perhaps merge the Java and native tarballs into one, but the build works, so if no one else is bothered by the oddity I think this is good to go.

        Show
        mackrorysd Sean Mackrory added a comment - So packages built on Debian are looking good too. I didn't make any substantial changes - I just rebased on top of the patch to add the zookeeper-rest package, and I removed my "TODO" note to perhaps merge the Java and native tarballs into one, but the build works, so if no one else is bothered by the oddity I think this is good to go.
        Hide
        rvs Roman Shaposhnik added a comment -

        Looks great! My last question is on what the package should be called. Shouldn't it be zookeeper-dev[el] to keep up with naming convention?

        Show
        rvs Roman Shaposhnik added a comment - Looks great! My last question is on what the package should be called. Shouldn't it be zookeeper-dev [el] to keep up with naming convention?
        Hide
        mackrorysd Sean Mackrory added a comment -

        I'm open to that. I just went with native because the idea was to pack up all the native bits of ZooKeeper, and this includes the shared libraries used at runtime and the native executables in addition to the static libraries and headers that would comprise a -dev[el] package. I suppose we could split them into multiple packages too...

        Show
        mackrorysd Sean Mackrory added a comment - I'm open to that. I just went with native because the idea was to pack up all the native bits of ZooKeeper, and this includes the shared libraries used at runtime and the native executables in addition to the static libraries and headers that would comprise a -dev [el] package. I suppose we could split them into multiple packages too...
        Hide
        rvs Roman Shaposhnik added a comment -

        I suppose the precedent here is hadoop-libhdfs-devel (RPM) and libhdfs0-dev (DEB). Sean Mackrory if its not too much to ask, would you mind changing the manes for ZK libraries accordingly?

        Show
        rvs Roman Shaposhnik added a comment - I suppose the precedent here is hadoop-libhdfs-devel (RPM) and libhdfs0-dev (DEB). Sean Mackrory if its not too much to ask, would you mind changing the manes for ZK libraries accordingly?
        Hide
        cos Konstantin Boudnik added a comment -

        Any progress here?

        Show
        cos Konstantin Boudnik added a comment - Any progress here?
        Hide
        rvs Roman Shaposhnik added a comment -

        Sean Mackrory ping! Any chance we can close this one?

        Show
        rvs Roman Shaposhnik added a comment - Sean Mackrory ping! Any chance we can close this one?
        Hide
        mackrorysd Sean Mackrory added a comment -

        Resolved as this was committed - I took you "would love to see this committed to trunk ASAP" as a +1. So if we have changes in that that we'd still like to see implemented I guess they can be done as separate JIRAs and I'll close this one out for now...

        Show
        mackrorysd Sean Mackrory added a comment - Resolved as this was committed - I took you "would love to see this committed to trunk ASAP" as a +1. So if we have changes in that that we'd still like to see implemented I guess they can be done as separate JIRAs and I'll close this one out for now...
        Hide
        mackrorysd Sean Mackrory added a comment -

        Just a note, I had committed this, but not pushed it. I just noticed the patch still sitting in my development branch for this feature. I just pushed it to master.

        Show
        mackrorysd Sean Mackrory added a comment - Just a note, I had committed this, but not pushed it. I just noticed the patch still sitting in my development branch for this feature. I just pushed it to master.

          People

          • Assignee:
            mackrorysd Sean Mackrory
            Reporter:
            rguo Guo Ruijing
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development