Flume
  1. Flume
  2. FLUME-1170

Create a framework for transaction listeners

    Details

    • Type: New Feature New Feature
    • Status: In Progress
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: v1.2.0
    • Fix Version/s: None
    • Component/s: Channel, Sinks+Sources
    • Labels:
      None

      Description

      Initially support on the sink side. It cannot be added to the channel as it is possible to support transaction listeners on both source and sink, and each side could be a different listeners. As of now, support on sink side transactions. It should be fairly easy to add this to source side, if we need to.

        Activity

        Hide
        Hari Shreedharan added a comment -

        Thanks for the comments Arvind.

        I thought about this and I decided it might be a good idea to add it on both sides, with configurability and functions that will be called either on put or take.
        Basically:

        • provide a way to configure if the callbacks should be made on put, get or both.
        • provide an interface with the following callback functions:
        • onPut(Map<String, String> eventHeaders), called on put, if the config specifies put or both.
        • onTake(Map<String, String> eventHeaders), called on take, based on config.
        • onCommit()
        • onRollback()

        I am adding support in MemoryChannel and RecoverableMemory channel and a unit test for MemoryChannel with txnlistener support.

        Show
        Hari Shreedharan added a comment - Thanks for the comments Arvind. I thought about this and I decided it might be a good idea to add it on both sides, with configurability and functions that will be called either on put or take. Basically: provide a way to configure if the callbacks should be made on put, get or both. provide an interface with the following callback functions: onPut(Map<String, String> eventHeaders), called on put, if the config specifies put or both. onTake(Map<String, String> eventHeaders), called on take, based on config. onCommit() onRollback() I am adding support in MemoryChannel and RecoverableMemory channel and a unit test for MemoryChannel with txnlistener support.
        Hide
        Arvind Prabhakar added a comment -

        Hari - thanks for filing this issue. What is the use-case of adding this on the sink side? It makes sense to have this bound only to transactions imo. If there is some aspect of functionality needed on the sink side, it should be implemented as a separate component.

        Show
        Arvind Prabhakar added a comment - Hari - thanks for filing this issue. What is the use-case of adding this on the sink side? It makes sense to have this bound only to transactions imo. If there is some aspect of functionality needed on the sink side, it should be implemented as a separate component.

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development