Details
Major Description
Currently the Fluo query observer framework (FQF) has a great plugin-in-ish way of doing exports. (See field: exporters in class: org.apache.rya.indexing.pcj.fluo.app.observers.QueryResultObserver ) Exports are query results and the action taken when they occur. Currently there are two. Despite this awesome framework, there are three interminglings that can become issues in a production system. Feel free to make sub tasks for each of these three tasks:
1. Create PCJ independently of alerts/triggers
At query register time (with FQF), optionally create PCJ's and optionally create alerts (with Alerting) preferably atomically, so if it fails in the middle it does not leave dangling references and garbage – currently there is one method that does everything: CreatePCJ.withRyaInteraction()
2. Enable exporters per query – currently all queries will be exported on all globally enabled exporters. When the query is registered (with FQF) specify the exporters that should be used.
3. Separate export observer queues per exporter – Currently retrying a failed export observer will re-export all export types, even if some already succeeded, causing duplicate triggers.
Also there is a need to refine the terms for java identifiers (for example: fields, classes) and documentation. Please document here a decision about what we name different things. The term PCJ are the index tables used by Rya to optimize it's queries. But sometimes it is used to describe the whole Fluo framework. Please define a unique term for the following. The starred term will be used here, but feel free to rename them:
- Rya query optimization PCJ (ex: the Blabla exporter) PCJ*
- Kafka trigger alert notifier (ex: the Foofoo exporter) Alerting*
- Fluo query observer framework (ex: results from the Barbar) FQF*
Unique identifers:
- FQF uses QueryID
- Alerting uses TopicID
- PCJ uses PcjId
Note that currently TopicID is assigned the same as QueryID, but that badly presupposes queryID's cannot be grouped onto one Topic.
