[STORM-2509] Writers in AbstractHDFSBolt are not closed/rotated when evicted from WritersMap

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: storm-hdfs
• Assignee: Unassigned
• Reporter: Ryan Persaud
• Created: 11/May/17 15:03
• Updated: 09/Jun/17 08:58

Description

When the eldest entry in the WritersMap in the AbstractHDFSBolt gets removed due to the number of writers exceeding maxWriters (see below), the writer is not closed and rotation actions are not executed (doRotationAndRemoveWriter is not called).

static class WritersMap extends LinkedHashMap<String, AbstractHDFSWriter> {
    final long maxWriters;

    public WritersMap(long maxWriters) {
        super((int)maxWriters, 0.75f, true);
        this.maxWriters = maxWriters;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry<String, AbstractHDFSWriter> eldest) {
        return this.size() > this.maxWriters;
    }
}

Issue Links

• links to: GitHub Pull Request #2119