Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14185

[SpannerIO.readChangeStreams] Drop metadata tables at the end of the job

Details

    • New Feature
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.37.0
    • 2.38.0
    • io-java-gcp
    • None

    Description

      The SpannerIO.readChangeStreams Connector uses metadata tables to keep track of its internal state during execution. At the moment, these metadata tables linger after the execution, meaning that users will have to drop them manually.

      In this change, we would like to drop them automatically once the job finishes. This should only occur after all partitions have been processed successfully and marked as finished.

      Attachments

        Activity

          People

            thiagotnunes Thiago Nunes
            thiagotnunes Thiago Nunes
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 50m
                2h 50m

                Slack

                  Issue deployment