Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Cannot Reproduce
-
None
-
None
-
None
Description
I am generating files with the following naming schems:
/host-data/metrics/$METRIC/$HOSTNAME/$MINUTE/$UNIXTIME
$METRIC is "ps", "nmon", "netstat", etc.
Using the following GetFile yml config:
Processors:
- name: GetNMon
class: org.apache.nifi.processors.standard.GetFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list: []
Properties:
Batch Size:
File Filter:
Ignore Hidden Files:
Input Directory: /host-data/metrics/nmon/HOSTNAME
Keep Source File:
Maximum File Age:
Maximum File Size:
Minimum File Age: 10 sec
Minimum File Size:
Path Filter:
Polling Interval:
Recurse Subdirectories:
In minifi-app.log.txt I see repeated entries for the $MINUTE folders, but no entries for the $UNIXTIME files underneath those $MINUTE folders.
I suspect the default "false" Keep Subdirectories value is causing MiNiFi to repeatedly send the $MINUTE folders and never progress into the files underneath.
Attachments
Issue Links
- Blocked
-
MINIFICPP-225 MiNiFi-cpp GetFile processor sends directories as files
- Resolved