Uploaded image for project: 'Apache ServiceComb'
  1. Apache ServiceComb
  2. SCB-1929

provide new mechanism: Filter

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • java-chassis-3.0.0
    • Java-Chassis
    • None

    Description

      currently, we have extensions:

      1. HttpClientFilter
      2. HttpServerFilter
      3. Handler
      4. ProducerInvokeExtension

      and we hard code to invoke all the extensions:

      1. consumer:
        1. invoke all handlers
        2. invoke all HttpClientFilters
      2. producer:
        1. schedule to operation related executor
        2. invoke all HttpServerFilters
        3. invoke all handlers

      PROBLEMS:

      1. all these extensions are about invocation, just have different prototype
      2. for consumer
        1. when a sync invocation failed, and want to retry, retry schedule is conflict to sync mechanism
      3. for producer
        1. qps-limiter should invoke before schedule, currently we hard code it
        2. if want to write reactive extension before schedule, to not care for controller method is sync or async, currently can not do it
        3. if executor is a real thread pool, and a reactive extension after schedule will break thread schedule plan, that maybe will cause block event loop
      4. exception convert only avaiable for controller method invoke, but all extensions exception need convert too

      SOLUTION:

      provide new mechanism: Filter, to unify all the extensions, and change main invocation process flow from hard code to by configuration

      1. prototype: 
        public @interface FilterMeta {
  String name();

  /**
   *
   * @return can be used for the specific invocation type
   */
  InvocationType[] invocationType() default {CONSUMER, PRODUCER};

  /**
   *
   * @return true to use same instance for each filter chain
   */
  boolean shareable() default true;
}

public interface Filter {
  CompletableFuture<Response> onFilter(Invocation invocation, FilterNode nextNode);
}
      2. configuration 
        servicecomb:
  filter-chains:
    enabled: true
    transport-filters:
      #default-consumer-transport:
      #  rest: rest-client-codec
      #  highway: highway-client-codec
      default-producer-tranport:
        rest: rest-server-codec
        highway: highway-server-codec
    consumer:
      default: simple-load-balance
      #default: simple-load-balance, default-consumer-transport, transport-client
      # samples for customize microservice filter chain
      #policies:
      #  ms-1: retry, load-balance, transport-client, ms-1-consumer-transport
    producer:
      default: default-producer-tranport, schedule, producer-operation
      # samples for customize microservice filter chain
      #policies:
      #  ms-1: qps-limiter, ms-1-producer-transport, schedule, producer-operation
        1. customize full flow of invocation
        2. microservice scope chain
        3. schema or operation scope chain, if have requirements in the future
        4. invoke exception converter when throwing a exception for all filter

      COMPATIBLE:

      1. wrap old HttpClientFilters to a Filter
      2. wrap old HttpServerFilters to a Filter
      3. wrap old consumer handlers to a Filter
      4. wrap old producer handlers to a Filter
      5. pre-define old filter chains, and default to old chains

      Attachments

        Issue Links

          Blocked

          Sub-task - The sub-task of the issue SCB-1938 OperationConfig can get in pool timeout by transport name

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1946 add new exception converter for all filters, not only the last one

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1947 define Filter and filter chain

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1948 cache invocation.producerArguments, avoid build one more time for validator

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1951 enhance HttpTransportContext

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1952 enhance HighwayTransportContext

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1955 create unify producer invocation flow

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1959 add rest invocation creator

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1961 add rest server codec filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1962 add highway server codec filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1965 add zipkin filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1966 print filter chain information after bootup

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1967 add simple load balance

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1968 add producer operation filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1979 load filter chains

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1980 add TransportFilters to select chain by transport in runtime

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1981 add filter chain to SCBEngine

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1982 add validator filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1985 allow producer invocation process by filter chain

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1987 adjust test code for filter chain mode

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-1992 tiny improve about InvocationException

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2029 tiny improve to make extend filter features easier

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2033 inject spring bean to filter instance automatically

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2034 move create validator to init to use injected spring bean

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2037 utils for get CompletableFuture result in sync mode

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2141 add RestClientTransportContext

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2142 add RestClientEncoder

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2146 add RestClientDecoder

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2147 add RestClientCodecFilter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2148 add RestClientSender

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2149 add RestClientSenderFilter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2150 allowed modify host before create HttpClientRequest

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2152 allow pojo invoker process by filter chain

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2156 add HighwayClientFilter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2158 optimize exception log

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2159 add TimeoutExceptionConverter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2169 optimize filter declare and chain config

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2170 add configuration to control log invocation exception stack trace

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2218 allow edge invocation process by filter

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SCB-2251 make exception processor extendable

          • Major - Major loss of function.
          • Resolved
          1.
          		 OperationConfig can get in pool timeout by transport name Sub-task Resolved Jimin Wu
          2.
          		 rename package core.filter to core.registry.discovery Sub-task Resolved Jimin Wu
          3.
          		 add new exception converter for all filters, not only the last one Sub-task Resolved Jimin Wu
          4.
          		 define Filter and filter chain Sub-task Resolved Jimin Wu
          5.
          		 cache invocation.producerArguments, avoid build one more time for validator Sub-task Resolved Jimin Wu
          6.
          		 enhance HttpTransportContext Sub-task Resolved Jimin Wu
          7.
          		 enhance HighwayTransportContext Sub-task Resolved Jimin Wu
          8.
          		 create unify producer invocation flow Sub-task Resolved Jimin Wu
          9.
          		 add rest invocation creator Sub-task Resolved Jimin Wu
          10.
          		 add rest server codec filter Sub-task Resolved Jimin Wu
          11.
          		 add highway server codec filter Sub-task Resolved Jimin Wu
          12.
          		 add zipkin filter Sub-task Resolved Jimin Wu
          13.
          		 print filter chain information after bootup Sub-task Resolved Jimin Wu
          14.
          		 add simple load balance Sub-task Resolved Jimin Wu
          15.
          		 add producer operation filter Sub-task Resolved Jimin Wu
          16.
          		 add documents of upgrade guide to tell how to migrate to new filter mechanism Sub-task Closed liubao
          17.
          		 load filter chains Sub-task Resolved Jimin Wu
          18.
          		 add TransportFilters to select chain by transport in runtime Sub-task Resolved Jimin Wu
          19.
          		 add filter chain to SCBEngine Sub-task Resolved Jimin Wu
          20.
          		 add validator filter Sub-task Resolved Jimin Wu
          21.
          		 allow producer invocation process by filter chain Sub-task Resolved Jimin Wu
          22.
          		 adjust test code for filter chain mode Sub-task Resolved Jimin Wu
          23.
          		 tiny improve about InvocationException Sub-task Resolved Jimin Wu
          24.
          		 tiny improve to make extend filter features easier Sub-task Resolved Jimin Wu
          25.
          		 add new invocation event to make extend features easier Sub-task Resolved Jimin Wu
          26.
          		 inject spring bean to filter instance automatically Sub-task Resolved Jimin Wu
          27.
          		 move create validator to init to use injected spring bean Sub-task Resolved Jimin Wu
          28.
          		 utils for get CompletableFuture result in sync mode Sub-task Resolved Jimin Wu
          29.
          		 add RestClientTransportContext Sub-task Resolved Jimin Wu
          30.
          		 add RestClientEncoder Sub-task Resolved Jimin Wu
          31.
          		 add RestClientDecoder Sub-task Resolved Jimin Wu
          32.
          		 add RestClientCodecFilter Sub-task Resolved Jimin Wu
          33.
          		 add RestClientSender Sub-task Resolved Jimin Wu
          34.
          		 add RestClientSenderFilter Sub-task Resolved Jimin Wu
          35.
          		 allowed modify host before create HttpClientRequest Sub-task Resolved Jimin Wu
          36.
          		 allow pojo invoker process by filter chain Sub-task Resolved Jimin Wu
          37.
          		 allow RestTemplate process by filter chain Sub-task Resolved Unassigned
          38.
          		 add HighwayClientFilter Sub-task Resolved Jimin Wu
          39.
          		 optimize exception log Sub-task Resolved Jimin Wu
          40.
          		 add TimeoutExceptionConverter Sub-task Resolved Jimin Wu
          41.
          		 optimize filter declare and chain config Sub-task Resolved Jimin Wu
          42.
          		 add configuration to control log invocation exception stack trace Sub-task Resolved Jimin Wu
          43.
          		 fix bug of ExceptionConverter throw exception Sub-task Resolved Jimin Wu
          44.
          		 allow edge invocation process by filter Sub-task Resolved Jimin Wu
          45.
          		 make exception processor extendable Sub-task Resolved Jimin Wu
          46.
          		 remove unused code for new filter Sub-task Resolved liubao

          Activity

            People

              liubao68 liubao
              wujimin Jimin Wu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: