[JENA-904] LPBRuleEngine leaks activeInterpreters

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Jena 3.0.0
• Component/s: Ontology API
• Assignee: Andy Seaborne
• Reporter: Stian Soiland-Reyes (old) (Inactive)
• Created: 21/Mar/15 01:29
• Updated: 29/Jul/15 14:52
• Resolved: 01/May/15 13:41

Description

I found that activeInterpreters in LPBRuleEngine leaks also if you don't iterate through to the end - calling .close() on the iterator is not enough. Clean-up seems to only happen when it.hasNext() is called and it returns false - so this could happen also in cases like where you return after getting the first hit.

	@Test
	public void testNotLeakingActiveInterpreters() throws Exception {
		Graph data = Factory.createGraphMem();
		data.add(new Triple(a, ty, C1));
		data.add(new Triple(b, ty, C1));
		List<Rule> rules = Rule
				.parseRules("[r1:  (?x p ?t) <- (?x rdf:type C1), makeInstance(?x, p, C2, ?t)]"
						+ "[r2:  (?t rdf:type C2) <- (?x rdf:type C1), makeInstance(?x, p, C2, ?t)]");

		FBRuleInfGraph infgraph = (FBRuleInfGraph) createReasoner(rules).bind(
				data);

		LPBRuleEngine engine = getEngineForGraph(infgraph);
		assertEquals(0, engine.activeInterpreters.size());
		assertEquals(0, engine.tabledGoals.size());

		// we ask for a non-hit -- it works, but only because we call it.hasNext()
		ExtendedIterator<Triple> it = infgraph.find(nohit, ty, C1);
		assertFalse(it.hasNext());
		it.close();
		assertEquals(0, engine.activeInterpreters.size());

		// and again.
		// Ensure this is not cached by asking for a different triple pattern
		ExtendedIterator<Triple> it2 = infgraph.find(nohit, ty, C2);
		// uuups, forgot to call it.hasNext(). But .close() should tidy
		it2.close();
		assertEquals(0, engine.activeInterpreters.size());

		
		// OK, let's ask for something that is in the graph
		
		ExtendedIterator<Triple> it3 = infgraph.find(a, ty, C1);
		assertTrue(it3.hasNext());
		assertEquals(a, it3.next().getMatchSubject());
		
		// .. and what if we forget to call next() to consume b?
		// (e.g. return from a method with the first hit)
		
		// this should be enough
		it3.close();
		// without leaks of activeInterpreters
		assertEquals(0, engine.activeInterpreters.size());
	}

When investigating this, I got as far as seeing that the activeInterpreters is normally closed from hasNext() when it reaches the end here:

https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/Generator.java#L303

which is a zombie check.. but this doesn't work when hasNext() hasn't reach the end, even though it is also called from close() here:

https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/LPTopGoalIterator.java#L199

Moving the checkForCompletions further down and calling .close() on any nextToRun or lookAhead was not sufficient - there is always a secondary Generator in the list which is not removed - only the top-level one is removed normally.

I was unable to investigate any further as I could not understand the classes that were creating the two Generators.

The inner LPInterpreter 147 is first created from generatorFor() call from ConsumerChoicePointFrame constructor within LPInterpreter.setupTabledCall which is coming from the ruleEngine.find.

This outer LPInterpreter is then added as part of the find().

In "normal operation" the inner one is removed through the zombie clearing, while the outer one is removed through .close()

If you don't iterate through to the end, the inner one is not removed.