[SLING-10208] JCR based implementation of the Distribution messaging API

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Content Distribution
• Assignee: Timothee Maret
• Reporter: Timothee Maret
• Created: 11/Mar/21 10:51
• Updated: 22/Feb/22 10:19

Description

Journal based Content Distribution persists messages in an append only journal of message. The interaction with the log go through a fairly small messaging API for which we currently ship an implementation for Apache Kafka.

The Kafka implementation is usable, for instance here but it requires a Kafka cluster which limits makes it harder to use in integration tests. This also prevent including the Journal based Content Distribution in the Apache Sling stater project.

To solve this issue, we could implement the messaging API (and thus the journal) exclusively on the repository.

How

The journal will be persisted on author and exposed to clients via a REST API. The journal could be backed by a "passive" Sling Job queue (an ORDERED queue without JobConsumer).

Monotonically increasing offsets could be implemented using Apache Oak Atomic Counter if and only we limit to non clustered author deployments.

Time based retention could be implemented by a periodical task that goes through the queue and remove old items.