Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      For use with ODBC drivers and to support binding to other languages, we should implement C++ client driver with C bindings.

      Key Features:

      • Zookeeper registry lookup
      • Connection management
      • Query submit
      • RecordBatch consumption

        Issue Links

          Activity

          Hide
          Jacques Nadeau added a comment -

          For building, I strongly suggest we treat it as a set of maven modules that drive cmake. I haven't used this particular integration: https://code.google.com/p/cmake-maven-project/ but something it should probably make things easier.

          Requirements for the C++ client should include compiling on OSX, Windows and Linux. We should include support for 32 and 64 bit environments.

          Show
          Jacques Nadeau added a comment - For building, I strongly suggest we treat it as a set of maven modules that drive cmake. I haven't used this particular integration: https://code.google.com/p/cmake-maven-project/ but something it should probably make things easier. Requirements for the C++ client should include compiling on OSX, Windows and Linux. We should include support for 32 and 64 bit environments.
          Hide
          Timothy Chen added a comment -

          +1 on maven, makes setup and CI a lot easier too

          Show
          Timothy Chen added a comment - +1 on maven, makes setup and CI a lot easier too
          Hide
          Xiao Meng added a comment -

          I am working on zookeeper registry lookup and have some questions about zookeeper registry.

          There are 3-nodes drillbits in my local machine, which are registered under the zookeeper path /drill/drillbits1
          I connect to zookeeper using its command line tool to get the DrillbitEndpoint information as following:

          [zk: localhost:2181(CONNECTED) 0] get /drill/drillbits1
          
          cZxid = 0x3
          ctime = Thu Jan 30 15:25:29 PST 2014
          mZxid = 0x3
          mtime = Thu Jan 30 15:25:29 PST 2014
          pZxid = 0x1a5
          cversion = 125
          dataVersion = 0
          aclVersion = 0
          ephemeralOwner = 0x0
          dataLength = 0
          numChildren = 3
          
          # I suppose this is the data entry name for drillbits
          [zk: localhost:2181(CONNECTED) 1] ls /drill/drillbits1
          
          [0a067c5c-1330-4919-89dc-b1ae5ce0c3f4, 9ec723a7-0571-4daa-8b42-039a7380e12e, 9e8824d6-0603-4ca1-9bae-de4a911abf4b]
          
          [zk: localhost:2181(CONNECTED) 2] get /drill/drillbits1/0a067c5c-1330-4919-89dc-b1ae5ce0c3f4
          
          $0a067c5c-1330-4919-89dc-b1ae5ce0c3f4�����(
                  localhost����
          cZxid = 0x1a3
          ctime = Wed Feb 19 12:00:16 PST 2014
          mZxid = 0x1a3
          mtime = Wed Feb 19 12:00:16 PST 2014
          pZxid = 0x1a3
          cversion = 0
          dataVersion = 0
          aclVersion = 0
          ephemeralOwner = 0x1444bba2f460003
          dataLength = 66
          numChildren = 0
          

          So my understanding is that each drillbit server will maintain a data entry for the DrillbitEndpoint information (address, port, and so on). To connect the drillbit, we need to retrieve the DrillbitEndpoint data from the zookeeper. I didn't find proper reference for the zookeeper integration in Drill.
          Can you explain more? Also how does the DrillbitEndpoint data encode/decode?

          Thanks.

          Show
          Xiao Meng added a comment - I am working on zookeeper registry lookup and have some questions about zookeeper registry. There are 3-nodes drillbits in my local machine, which are registered under the zookeeper path /drill/drillbits1 I connect to zookeeper using its command line tool to get the DrillbitEndpoint information as following: [zk: localhost:2181(CONNECTED) 0] get /drill/drillbits1 cZxid = 0x3 ctime = Thu Jan 30 15:25:29 PST 2014 mZxid = 0x3 mtime = Thu Jan 30 15:25:29 PST 2014 pZxid = 0x1a5 cversion = 125 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 3 # I suppose this is the data entry name for drillbits [zk: localhost:2181(CONNECTED) 1] ls /drill/drillbits1 [0a067c5c-1330-4919-89dc-b1ae5ce0c3f4, 9ec723a7-0571-4daa-8b42-039a7380e12e, 9e8824d6-0603-4ca1-9bae-de4a911abf4b] [zk: localhost:2181(CONNECTED) 2] get /drill/drillbits1/0a067c5c-1330-4919-89dc-b1ae5ce0c3f4 $0a067c5c-1330-4919-89dc-b1ae5ce0c3f4�����( localhost���� cZxid = 0x1a3 ctime = Wed Feb 19 12:00:16 PST 2014 mZxid = 0x1a3 mtime = Wed Feb 19 12:00:16 PST 2014 pZxid = 0x1a3 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x1444bba2f460003 dataLength = 66 numChildren = 0 So my understanding is that each drillbit server will maintain a data entry for the DrillbitEndpoint information (address, port, and so on). To connect the drillbit, we need to retrieve the DrillbitEndpoint data from the zookeeper. I didn't find proper reference for the zookeeper integration in Drill. Can you explain more? Also how does the DrillbitEndpoint data encode/decode? Thanks.
          Hide
          Timothy Chen added a comment -

          The DrillBitEndPoint is a Protobuf message, and we simply write and read the binary encoding of that message.
          The SerDe is in DrillbitEndpointSerDe.java.

          Show
          Timothy Chen added a comment - The DrillBitEndPoint is a Protobuf message, and we simply write and read the binary encoding of that message. The SerDe is in DrillbitEndpointSerDe.java.
          Hide
          Xiao Meng added a comment -

          Hi, I've put the current work on github https://github.com/xiaom/incubator-drill/tree/drillclient/contrib/native/client. It is pretty much a prototype, so bear with the smell. It also included a sample program to submit query and get response from one Drillbit. I've tested it on a CentOS 6.5 box. The aync I/O part is not quiet ready and we need to work it out.

          Show
          Xiao Meng added a comment - Hi, I've put the current work on github https://github.com/xiaom/incubator-drill/tree/drillclient/contrib/native/client . It is pretty much a prototype, so bear with the smell. It also included a sample program to submit query and get response from one Drillbit. I've tested it on a CentOS 6.5 box. The aync I/O part is not quiet ready and we need to work it out.
          Hide
          Xiao Meng added a comment -

          So how does zookeeper choose the proper drillbit to run the query for the client? Does it do some load balance job? I am just wondering whether zookeeper provide C API to choose proper drillbit for the client.

          Show
          Xiao Meng added a comment - So how does zookeeper choose the proper drillbit to run the query for the client? Does it do some load balance job? I am just wondering whether zookeeper provide C API to choose proper drillbit for the client.

            People

            • Assignee:
              Parth Chandra
              Reporter:
              Jacques Nadeau
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development