Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1714

Creating shared context factory for shared context objects

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0
    • None
    • None

    Description

      Currently, ContextManager only provides a single scope, single object shared context in the high-level API:

      
      public interface ContextManager {
         public void init(Config config, TaskContext context);
      }

      It is not sufficient to address the following requirements:
      1) different scope of shared objects (i.e. operator-scope, task-scope, container-scope)
      2) different shared objects instantiated and shared in the same scope (i.e. user program has task-level shared objects for remote-http-client vs in-mem cache that needs to managed independently)

      An initial idea to implement the SharedContextFactory is the following:

      public interface SharedContextFactory {
          <T extends SharedContext> T createSharedContext(Config config, Scope scope, final SystemContext sysContext, String id);
      }
      
      public interface SharedContext {
        void init(Config, final SystemContext sysContext);
        void close();
      }
      
      public interface SystemContext {
        <T extends SharedContext> T getSharedContext(Scope scope, String id);
      }
      
      public interface SamzaContainerContext extends SystemContext {
      }
      
      public interface TaskContext extends SystemContext {
      }
      

      Note that this is also needed to serialize operator functions used in high-level API that depends on non-serializable third-party libraries (e.g. Calcite runtime objects used in Samza SQL).

      Attachments

        Issue Links

          Activity

            People

              cameronlee Cameron Lee
              nickpan47 Yi Pan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: