Uploaded image for project: 'Apache InLong'
  1. Apache InLong
  2. INLONG-3

C++ SDK support in TubeMQ

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • None
    • 0.6.0
    • None

    Description

      I will contribute the SDK of C + +, which is currently being sorted out. Without changing the server-side interaction protocol, the semantics will be the same as the Java implementation.


      Task
      1.io thread management
      2. Connection interface
      3. Connection pool management
      4.IO Buffer
      5.Future/Promise support
      6.Codec interface and TubeMQ interactive encoding
      7. Functionalization of service interface parameter settings
      8.Future service interface
      9.Master implementation, metadata acquisition and reading interface
      10. Consumer realization
      11. Producer Realization
      12.Client configuration
      13. Producer API package
      14. Consumer API package

      Plan
      Phase 1 (Available)-Implement basic functions and ensure unit test coverage
      RPC support
      Message consumption (pull mode)
      Write message (synchronous)
      Connection pool implementation-automatic reconnection of broken links, automatic recovery of idle connections, shared according to sessionFactory
      Test case and specification construction-

      Phase 2 (Practical)-Realize the features of each function and be able to adapt to a certain level of pressure measurement
      Asynchronous production 
      Authentication and authentication && Prevent production and consumption around the master 
      Accurately specify offset partition consumption 
      Frequency control
      Consumption of multiple topics in a single group 
      Server filter consumption 

      Phase 3 (Easy to use)-Continuous iteration, full-featured implementation, realizing the latest function points according to priority
      Inactive for more than specified minutes, mainly at the production end, such as 3 minutes 
      Automatically shield the dead pixels, detect the dead pixels through the algorithm, and automatically shield the fault Broker's data transmission 
      TLS 
      push consumption

       

      Featrue
      1.rpc development
      Pre-research--asio development
      Connection interface
      Connection pool management-connection reuse, automatic reconnection of broken links, automatic recovery of idle connections, sharing according to sessionFactory
      Streaming request support-based on serialNo
      Future/Promise interface
      Codec interface
      Log
      Config
      Buffer

      2. Service interface development
      API service interface, such as heartbeat package variable function
      Request Future/Promise function encapsulation
      Config: client, consumer, producer, Master, rpc
      Message id

      3.Client
      turn stop
      Configuration input
      Consumer API
      Producer API

      4.Master
      Authentication
      Prevent the production and consumption around the Master: get the token from the Master and update it to the local metadata
      Metadata management: Topic, Group, Master
      Master Heartbeat

      5. Consumer
      User interface
      Heartbeat support
      pull consumption
      Consumption of multiple topics in a single group
      Authentication
      Precisely specify offset partition consumption

      6. Producer
      User interface
      Simultaneous production
      Asynchronous production
      Load balancing algorithm: polling
      Load balancing algorithm: hash

      7. Advanced features
      Master disconnects and reconnects, traversing dns to obtain Master host IP
      Limiting
      Effectively-Once
      tls:asio+openssl
      Filter consumption
      Inactive for more than the specified minutes, mainly at the production end, for example 3 minutes
      Fuse: Automatically shield the dead spots, detect the dead spots through the algorithm, automatically shield the fault Broker's data transmission
      Reporting time-consuming, failure; static weight (number of partitions), dynamic weight
      Connection pool management-automatic recovery of idle connections, shared according to sessionFactory
      Push consumptio

       

       

      Attachments

        1. Tubemq-cpp-client.png
          29 kB
          Haiji Li
        1.
        Create C/C++ configure files Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 50m
        2.
        Create C/C++ Codec utils Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        3.
        Create C/C++ Metadata classes Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        4.
        Create C++ flow control handler Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        5.
        Create C/C++ ini file read utils Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        6.
        [TUBEMQ-266] Add Tencent/rapidjson as submodule Sub-task Resolved Guangxu Cheng

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        7.
        Create C/C++ Message class Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        8.
        C++ SDK log module Sub-task Resolved Haiji Li

        3%

        Original Estimate - 48h
        Time Spent - 1h 50m Remaining Estimate - 46h 10m
        9.
        Create C/C++ RmtDataCache class Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 2h
        10.
        this point c++ SDK class Sub-task Resolved Haiji Li

        0%

        Original Estimate - 48h
        Remaining Estimate - 47h 40m
        11.
        C++ SDK copy constructor and assignment constructor Sub-task Resolved Haiji Li

        1%

        Original Estimate - 24h
        Time Spent - 20m Remaining Estimate - 23h 40m
        12.
        Unified C/C++ files's code style Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 10m
        13.
        C++ SDK dir name change inc -> include/tubemq/ Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        14.
        Support CMake compilation Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 20m
        15.
        Thread Pool & Timer Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        16.
        Create C/C++ Unit Tests Sub-task Closed Unassigned

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        17.
        Create C/C++ subscribe info class Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        18.
        atomic_def.h use C++11 stdlib class Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        19.
        Create C/C++ return result class Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        20.
        Adjust C/C++ some file names: add "tubemq_" prefix Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        21.
        Replace C/C++ pthread's mutex to std::mutex Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        22.
        Create C/C++ SDK's manager class Sub-task Resolved Guocheng Zhang

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 20m
        23.
        C++ SDK io buffer Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 10m
        24.
        C++ SDK Codec interface Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 20m
        25.
        C++ SDK Codec TubeMQ proto support Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        26.
        C++ SDK TCP Connect Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        27.
        C++ SDK Connect Pool Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 50m
        28.
        C++ SDK singleton & executor_pool optimization Sub-task Resolved Haiji Li

        2%

        Original Estimate - 24h
        Time Spent - 40m Remaining Estimate - 23h 20m
        29.
        C++ SDK Create Future class Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1.5h
        30.
        C++ SDK Create client API Sub-task Resolved Haiji Li

        1%

        Original Estimate - 48h
        Time Spent - 40m Remaining Estimate - 47h 20m
        31.
        C++SDK Client handler detail Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        32.
        C++ SDK Create Thread Pool Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        33.
        C++ SDK client code adj Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        34.
        C++ SDK example&tests Sub-task Resolved Haiji Li

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m

        Activity

          People

            charleli Haiji Li
            zhangguocheng zhangguocheng
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 192h Original Estimate - 192h
                192h
                Remaining:
                Time Spent - 32h 40m Remaining Estimate - 188h 10m
                188h 10m
                Logged:
                Time Spent - 32h 40m Remaining Estimate - 188h 10m
                32h 40m