Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.20.0
-
None
Description
In our environment, we use ListenUDPRecord to collect Syslog messages. This processor is followed by a PartitionRecord processor that populates an attribute for routing.In release 1.19.0, this flow worked without issue. In 1.20.0 though, I am seeing intermittent message truncation in PartitionRecord, with bulletin messages like the following appearing regularly:
PartitionRecord[id=03ea67a7-0b9c-1c9f-ffff-ffff8d7e5185] Failed to partition FlowFile[filename=ca9c3e11-9365-4ff9-9499-29522fc0cab7]: com.fasterxml.jackson.core.JsonParseException: Unexpected character (',' (code 44)): expected a value
at [Source: (org.apache.nifi.stream.io.NonCloseableInputStream); line: 1, column: 381]
An example message (note the absence of a Syslog header):
itor] [Unit test] Alarm check cfg warning threshold=75 critical threshold=85 warning alarm <...>
ListenUDPRecord properties are attached.
Reproduction
The attached minimal flow illustrates this setup.
To reproduce this issue, generate improperly-formatted syslog and send to ListenUDPRecord.
In my environment, I have two syslog sources feeding this test cluster. Scenario is as follows:
- First source (compliant Syslog format) feeds in.
- Flow is OK - no bulletins.
- Activate second source, which is of an invalid Syslog format and flows to the parse.failure relationship of ListenUDPRecord. This is expected - I deal with this gracefully.
- Bulletins start firing in PartitionRecord and the first source starts getting truncated randomly.
Overall, the majority of messages from the well-formed source make it through. However I'm seeing roughly 1 bulletin every few seconds, which indicates a small proportion of messages are getting truncated.
Attachments
Attachments
Issue Links
- is related to
-
NIFI-10887 Improve Performance of ReplaceText processor
-
- Resolved
-
- links to
