[APEXMALHAR-2407] Time buckets are not purging after expiry

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.7.0
• Assignee: Chaitanya Chebolu
• Reporter: Chaitanya Chebolu
• Created: 13/Feb/17 12:13
• Updated: 01/Apr/17 05:51
• Resolved: 19/Mar/17 14:23

Description

Below are the observations:
1) purgeListner is not triggering purgeTimeBucketsLessThanEqualTo() in MovingTimeBucketAssigner.
This will trigger only if the value of "triggerPurge" is true.
"triggerPurge" calculated based on (diffInBuckets > 0) condition. "diffInBuckets" is calculated based on below condition:
if (time >= end)

{ long diffInBuckets = (time - end) / bucketSpanMillis; ------ }

If the data is processing time then the value of diffInBuckets is always zero.
2) Comparing the key bucket Id with time bucket while writing bucketed data which was happening in transferWindowFiles() which is in IncrementalCheckpointManager.