Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-10605

fix error in 32bit jvm object alignment gap calculation

Details

    • Bug
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • 8.11.1
    • 9.3
    • core/other
    • None
    • jdk 7 32-bit

      jdk 8 32-bit

    • New

    Description

      ArrayUtil.oversize(int minTargetSize, int bytesPerElement)
      This method is used to calculate the optimal length of an array during expansion.
       
      According to current logic,in order to avoid space waste caused by object alignment gap. In 32-bit JVM,the array length will select the numbers(the current optional columns) in the table below. But the results weren't perfect.

      For example, if I want to expand byte[2], I will call the method oversize(2,1) to get the size of the next array, which returns 8.
      But byte [8] is not the best result.
      Since byte[8] and byte[12] use the same memory space (both are 24 bytes due to alignment gap),
      So it's best to return 12 here.
      See the table below.

       
      I used jol-core to calculate object alignment gap

      <dependency>
          <groupId>org.openjdk.jol</groupId>
          <artifactId>jol-core</artifactId>
          <version>0.16</version>
          <scope>compile</scope>
      </dependency> 

       
      Execute the following code:

      System.out.println(ClassLayout.parseInstance(new byte[6]).toPrintable()); 

       

       
      To further verify that the tool's results are correct, I wrote the following code to infer how much space the array of different lengths actually occupies based on when the OOM occursThe conclusion is consistent with jol-core.

      // -Xms16m -Xmx16m
      // Used to infer the memory space occupied
      // by the length of various arrays
      public static void main(String[] args) {
          byte[][] arr = new byte[1024 * 1024][];
          for (int i = 0; i < arr.length; i++) {
              if (i % 100 == 0) {
                  System.out.println(i);
              }
              // According to OOM occurrence time
              // in 32-bit JVM,
              // Arrays range in length from 5 to 12,
              // occupying the same amount of memory
              arr[i]=new byte[5];
          }
      } 

      new byte[5] and new byte[12] use the same amount of memory


       

      In addition - XX: ObjectAlignmentInBytes should also affect the return value of this method. But I don't know whether it is necessary to do this function. If necessary, I will modify it together. Thank you very much!

       

      Attachments

        1. image-2022-06-08-20-50-27-712.png
          16 kB
          sun wuqiang
        2. image-2022-06-08-21-24-57-674.png
          48 kB
          sun wuqiang
        3. image-2022-06-09-08-25-55-289.png
          2 kB
          sun wuqiang
        4. image-2022-06-09-08-26-36-528.png
          16 kB
          sun wuqiang

        Activity

          People

            Unassigned Unassigned
            wormday sun wuqiang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 20m
                3h 20m