Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-212

Explore using PostgreSQL changelog as input stream

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None

    Description

      It would be good to be able to capture changes to a PostgreSQL database, and use it as an input to Samza (like SAMZA-200 but for Postgres instead of MySQL).

      On a high level, it seems like the options are:

      @selenamarie points out: "problem with it is that LISTEN/NOTIFY are stateless. if you want reliable transport, will need logical replication".

      This Twitter conversation has some more detail.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            martinkl Martin Kleppmann
            martinkl Martin Kleppmann
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment