[NIFI-6339] ListHDFS processor will list files without using previous state when cluster startup

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0, 1.9.1, 1.9.2
• Fix Version/s: 1.10.0
• Component/s: Core Framework
• Assignee: Mark Payne
• Reporter: Hsin-Ying Lee
• Created: 31/May/19 07:24
• Updated: 09/Aug/19 17:08
• Resolved: 12/Jun/19 15:00

Description

When the node startup, NiFi will create processor and load the property & state.

But ListHDFS will ignore the previous listed-state stored on Zookeeper, and relist all file again.

 

And I found when we call the function setProperty, we only check the value between default and newValue. If oldValue is same with newValue, it'll also trigger function onPropertyModified.

It casue ListHDFS loaclVariable: resetState to be true. When ListHDFS be triggered, it'll clear the state empty, and relist all the files in the directory.

 

// AbstractComponentNode.java

if (!value.equals(propertyModComparisonValue)) {
    try {
        onPropertyModified(descriptor, oldValue, value);
    } catch (final Exception e) {
        // nothing really to do here...
    }
}

 

 

Issue Links

• is duplicated by: NIFI-6533 listHDFS lists files from beginning after restart NIFI
• links to: GitHub Pull Request #3530