Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15446

Per-thread stack size is too small on aarch64 CentOS

    XMLWordPrintableJSON

    Details

    • Bug Category:
      Code - Bug - Unclear Impact
    • Severity:
      Normal
    • Complexity:
      Low Hanging Fruit
    • Discovered By:
      User Report
    • Platform:
      Java8, Java11, OpenJDK, Linux, ARM

      Description

      Hi all,

      I found an issue when I tried to start cassandra on my aarch64 CentOS7.6, however no errors on Ubuntu. Of course I could increase -Xss in jvm.options to fix it, but this issue also caused Cassandra's docker images from docker hub could not run containers on this OS.

      The information of my current environment and root cause of this issue were shown below.

      Error

      The stack size specified is too small, Specify at least 328k
      Error: Could not create the Java Virtual Machine.
      Error: A fatal exception has occurred. Program will exit.

      Version

      Cassandra 2.1.21 2.2.15 3.0.19 3.11.5 

      Environment

      $ lscpu

      Architecture: aarch64
      Byte Order: Little Endian

      $ uname -m

      aarch64

      $ java -version

      openjdk version "1.8.0_181"
      OpenJDK Runtime Environment (build 1.8.0_181-b13)
      OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
      [root@localhost apache-cassandra-3.11.5]# cat /etc/os-release

      $ cat /etc/os-release

      NAME="CentOS Linux"
      VERSION="7 (AltArch)"
      ID="centos"
      ID_LIKE="rhel fedora"
      VERSION_ID="7"
      PRETTY_NAME="CentOS Linux 7 (AltArch)"
      ANSI_COLOR="0;31"
      CPE_NAME="cpe:/o:centos:centos:7"
      HOME_URL="https://www.centos.org/"
      BUG_REPORT_URL="https://bugs.centos.org/"

      Root Cause

      Checked openjdk-1.8.0 source code, the min stack size is calculated by StackYellowPage, StackRedPage, StackShadowPage, OS page size. Among those parameters, default OS page size of aarch64 CentOS 7.6 is 64K, however aarch64 Ubuntu 18.04 and X86 CentOS are both 4K.

      This difference causes JVM on aarch64 Ubuntu 18.04 needs 164K per-thread stack size, but 328K required on aarch64 CentOS 7.6.

      The formula is 

      os::Linux::min_stack_allowed = MAX2(os::Linux::min_stack_allowed,
      (size_t)(StackYellowPages+StackRedPages+StackShadowPages) * Linux::page_size() +
      (2*BytesPerWord COMPILER2_PRESENT(+1)) * Linux::vm_default_page_size());

      Parameters on aarch64 CentOS7.6

      intx StackRedPages = 1 
      intx StackShadowPages = 1 
      intx StackYellowPages = 1 

      pageSize 64K

      BytesPerWord 8

      vm_default_page_size 8K

      As a result, we have min_stack_allowed = (1 + 1 + 1) * 64K + (2 * 8 + 1) * 8K = 328K

       

      I could see some similar issues asked for specified achitecture, but no root cause analyzed. I hope this could help you decide proper stack size for all common OS.

      If you have any suggestion, pls let me know.

       

        Attachments

          Activity

            People

            • Assignee:
              maaaace Heming Fu
              Reporter:
              maaaace Heming Fu
              Authors:
              Heming Fu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: