[BAHIR-72] support netty: pushed tcp/http connector

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Affects Version/s: Flink-1.0
• Fix Version/s: Flink-1.0
• Component/s: Flink Streaming Connectors
• Assignee: shijinkui
• Reporter: shijinkui
• Created: 24/Oct/16 10:37
• Updated: 23/May/17 18:40
• Resolved: 23/Nov/16 10:46

Description

Also discuss from flink: https://issues.apache.org/jira/browse/FLINK-4630

When source stream get start, listen a provided tcp port, receive stream data from user data source.
This netty tcp source is keepping alive and end-to-end, that is from business system to flink worker directly.

Such source service is needed in produce indeed.

describe the source in detail below:

1. source run as a netty tcp and http server
2. user provide a tcp port, if the port is in used, increace the port number between 1024 to 65535. Source can parallel.
3. callback the provided url to report the real port to listen
4. user push streaming data to netty server, then collect the data to flink

Issue Links

• is related to: BAHIR-113 Flink Netty connector missing automated unit tests
• relates to: FLINK-4630 add netty tcp/restful pushed source support
• links to: GitHub Pull Request #7