[FLUME-2653] Allow inUseSuffix to be null/empty - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.5.1
Fix Version/s: 1.9.0
Component/s: Sinks+Sources
Labels:
- docs-missing
- hdfssink

Release Note:
Needs documentation update for the additional parameter.

Description

At the moment, it doesn't seem possible to set the null/empty. We've tried '' which just adds the quotes to the end, and setting to nothing, which just uses the default .tmp.

We want the in use file to have the same name as the closed file, so we can read from files that are in use without the file moving from underneath us. In our use case, we know that an in use file is still readable and parseable, because it is just text with a JSON document per line.

It looks like the HDFS sink code can handle this change already, but at the moment there is no way to set the bucketPath and targetPath to be the same.