[FLUME-2623] Test testRemoveFields(org.apache.flume.source.TestSyslogUdpSource) fails randomly

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.5.0
• Component/s: Test

Environment

RHEL 7.1 on PPC64 LE

• Assignee: Unassigned
• Reporter: Tony Reix
• Created: 12/Feb/15 15:18
• Updated: 30/Sep/16 08:49

Description

The test org.apache.flume.source.TestSyslogUdpSource is not 100% reliable. It fails sometimes randomly.

Source code dealing with the issue is:
flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java (about line 101)

for (int i = 0; i < 100 ; i++)

{ syslogSocket = new DatagramSocket(); syslogSocket.send(datagramPacket); syslogSocket.close(); }

List<Event> channelEvents = new ArrayList<Event>();
Transaction txn = channel.getTransaction();
txn.begin();
for (int i = 0; i < 100; i++)

{ Event e = channel.take(); Assert.assertNotNull(e); <<<<<<<<<<<< channelEvents.add(e); }

Sometimes... "e" is null.

Failure deals with:
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526) at org.junit.Assert.assertNotNull(Assert.java:537)
at org.apache.flume.source.TestSyslogUdpSource.runKeepFieldsTest(TestSyslogUdpSource.java:101)
at org.apache.flume.source.TestSyslogUdpSource.testRemoveFields(TestSyslogUdpSource.java:177)

With OpenJDK, I got it failing once out of 30 tries.
However, with IBM JVM, I got if failing 6 times out of 10.

After I added a Thread.sleep(2000) in the middle, with IBM JVM, I've reduced the probability of the failure from 6/10 to 2/10 . So, that helps, but that is not enough. A better solution must be found.

The issue appears more often with IBM JVM probably because things are handled differently, or quicker, by IBM JVM. Anyway, the issue also appears with OpenJDK.

I guess that the issue is still there with master version.