[FLUME-3268] Introducing micro batch processing to HDFSEventSink - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
None

Description

In our test with HDFSEvent sink, we found that we could increase the draining speed of HDFSSink up to 4x by introducing micro batch processing. With the micro batch processing feature, we will batch the events written to HDFS instead of one by one.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

FLUME-3268-0.patch
13/Aug/18 19:46
22 kB
zhenzhao wang

Issue Links

links to

GitHub Pull Request #223

Activity

People

Assignee:: Unassigned

Reporter:: zhenzhao wang

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 13/Aug/18 19:43

Updated:: 15/Aug/18 18:13