[GSOC-126] GSoC: Observability Improvement for RocketMQ Streams

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: RocketMQ full-time gsoc2023 mentor
• Assignee: Unassigned
• Reporter: Ni Ze
• Created: 20/Mar/23 02:05
• Updated: 21/Mar/23 08:08

Description

RocketMQ Streams 

RocketMQ Streams is a lightweight stream processing framework, application gains the stream processing ability by depending on RocketMQ Streams as an SDK.

Background

Repo of RocketMQ Streams:  https://github.com/apache/rocketmq-streams
The architecture document of RocketMQ Streams: RocketMQ Streams examples,

architecture doc

Task

The observability needs to be enhanced in the following aspects:

• The metric of client/processor/thread/state/rocksdb;
• The topology of streaming process;
• Mutli-output of metrics;

This task need you to study the implementation details of RocketMQ Streams streams, and find out the key indicators in the stream processing process. Design and implement a complete set of observability solutions, and finally use it to complete runtime problem diagnosis;

Mentor

nize, Committer of of Apache RocketMQ, [e-mail]