Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-41318

Validate scalability improvements due to JDK-8180450 on Netty 4.1.84.Final

    XMLWordPrintableJSON

Details

    Description

      https://netty.io/news/2022/10/11/4-1-84-Final.html has introduced a bug-fix/improvement to prevent a JDK scalability issue due to type check against interfaces (see https://github.com/netty/netty/pull/12806 for more details - and https://redhatperf.github.io/post/type-check-scalability-issue/ for a deeper and wider analysis).

      it seems that Spark network make uses of Netty channel duplex handlers on TransportContext and it's likely to be affected by it (or better, depending which netty version).

      Moreover, I suggest to verify if there are other places where the issue can happen and eventually, report them on https://bugs.openjdk.org/browse/JDK-8180450 as well (if they use type checks in an idiomatic form).
      The blog post has linked https://github.com/RedHatPerf/type-pollution-agent to detect dangerous candidate code paths.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nigrofranz Francesco Nigro
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: