[NIFI-8727] claimantCount will never decrement to zero

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.12.0, 1.13.0, 1.12.1, 1.13.1, 1.13.2
• Fix Version/s: 1.15.0
• Component/s: Core Framework
• Environment: linux
• Flags: Important
• Assignee: Mark Bean
• Reporter: wangliqiang
• Created: 23/Jun/21 01:11
• Updated: 07/Nov/21 00:08
• Resolved: 29/Sep/21 18:04

Description

When running my processor code below :

//originalFlowFile has content , so ClaimantCount=1
 FlowFile multiFlowFile = session.clone(originalFlowFile); // claim count +1，so ClaimantCount=2
 multiFlowFile = session.write(multiFlowFile, new OutputStreamCallback() {
     @Override
     public void process(OutputStream out) throws IOException {
        IOUtils.write(tvMultiAlbumJson, out, Charset.forName("UTF-8"));
     }
 });//the new content will resuse the same resource claim , so ClaimantCount=3
 //At this point we have two flowfile and two contentClaim ，and ClaimCount=3.
 //When this two flowfiles dropped，the claimantCount should decrement to 0，however the result is ClaimantCount=1！
 //If we use "sh nifi.sh diagnostics --verbose dump.log" to get a dump log，we will find some info like this “default/465/1623853427574-10705, Claimant Count = 1, In Use = true, Awaiting Destruction = false, References (0) = []” 
 //And the file “default/465/1623853427574-10705” will never be archived，and will never be destroyed，and the content_repository will use more storage than it configs.

The above is a sort of phenomenon. The reason is the code below：

//session.clone
 public FlowFile clone(FlowFile example, final long offset, final long size) {
     .....................................
     final StandardRepositoryRecord record = new StandardRepositoryRecord(null); //here the originalFlowFileRecord of record is null
     .....................................
     return clone;
 }
 //session.commit
 private void updateClaimCounts(final RepositoryRecord record) {
     ..........................................................
     if (record.isContentModified()) {
     decrementClaimCount(originalClaim); //here the originalClaim is null
     }
 }

Perhaps we should not use session.clone like that，but without official note it will sometimes happen to be used.
So i change "final StandardRepositoryRecord record = new StandardRepositoryRecord(null)" to "final StandardRepositoryRecord record = new StandardRepositoryRecord(null, currRec);"

 

 

Issue Links

• fixes: NIFI-9056 Content Repository Filling Up
• links to: GitHub Pull Request #5176
• links to: GitHub Pull Request #5418

Sub-Tasks

• archiveCount below zero, all processors with content modified will blocked in "waiting for archive cleanup": Resolved