Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4, 6.0
    • Component/s: Response Writers, SolrCloud
    • Labels:
      None

      Description

      As discussed in SOLR-7927, we can reduce the buffer memory allocated by JavaBinCodec while writing large strings.

      https://issues.apache.org/jira/browse/SOLR-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700420#comment-14700420

      The maximum Unicode code point (as of Unicode 8 anyway) is U+10FFFF (http://www.unicode.org/glossary/#code_point). This is encoded in UTF-16 as surrogate pair \uDBFF\uDFFF, which takes up two Java chars, and is represented in UTF-8 as the 4-byte sequence F4 8F BF BF. This is likely where the mistaken 4-bytes-per-Java-char formulation came from: the maximum number of UTF-8 bytes required to represent a Unicode code point is 4.

      The maximum Java char is \uFFFF, which is represented in UTF-8 as the 3-byte sequence EF BF BF.

      So I think it's safe to switch to using 3 bytes per Java char (the unit of measurement returned by String.length()), like CompressingStoredFieldsWriter.writeField() does.

      1. SOLR-7971.patch
        1 kB
        Shalin Shekhar Mangar
      2. SOLR-7971-directbuffer.patch
        4 kB
        Shalin Shekhar Mangar
      3. SOLR-7971-directbuffer.patch
        4 kB
        Shalin Shekhar Mangar
      4. SOLR-7971-directbuffer.patch
        3 kB
        Shalin Shekhar Mangar
      5. SOLR-7971-doublepass.patch
        5 kB
        Shalin Shekhar Mangar
      6. SOLR-7971-doublepass.patch
        4 kB
        Noble Paul
      7. SOLR-7971-doublepass.patch
        4 kB
        Shalin Shekhar Mangar

        Activity

        Hide
        Shalin Shekhar Mangar added a comment -

        Without this patch, indexing the same 100MB JSON document mentioned in SOLR-7927:

        1. succeeds on ./bin/solr start -m 2100M
        2. fails on ./bin/solr start -m 2000M

        And with this change:

        1. succeeds on ./bin/solr start -m 1900M with patch
        2. fails on ./bin/solr start -m 1800M with patch
        Show
        Shalin Shekhar Mangar added a comment - Without this patch, indexing the same 100MB JSON document mentioned in SOLR-7927 : succeeds on ./bin/solr start -m 2100M fails on ./bin/solr start -m 2000M And with this change: succeeds on ./bin/solr start -m 1900M with patch fails on ./bin/solr start -m 1800M with patch
        Hide
        ASF subversion and git services added a comment -

        Commit 1697726 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1697726 ]

        SOLR-7971: Reduce memory allocated by JavaBinCodec to encode large strings by an amount equal to the string.length()

        Show
        ASF subversion and git services added a comment - Commit 1697726 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1697726 ] SOLR-7971 : Reduce memory allocated by JavaBinCodec to encode large strings by an amount equal to the string.length()
        Hide
        ASF subversion and git services added a comment -

        Commit 1697727 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1697727 ]

        SOLR-7971: Reduce memory allocated by JavaBinCodec to encode large strings by an amount equal to the string.length()

        Show
        ASF subversion and git services added a comment - Commit 1697727 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1697727 ] SOLR-7971 : Reduce memory allocated by JavaBinCodec to encode large strings by an amount equal to the string.length()
        Hide
        Shalin Shekhar Mangar added a comment -

        Here's another idea as a patch to further reduce heap requirement. In this patch I use a direct byte buffer to hold the encoded bytes and limit the intermediate on-heap buffer to 64KB only. This optimization kicks in only if the max bytes required by the string being serialized is greater than 64KB.

        With this patch I can index the same 100MB JSON document with 1200MB of heap.

        Yonik Seeley - Thoughts?

        Show
        Shalin Shekhar Mangar added a comment - Here's another idea as a patch to further reduce heap requirement. In this patch I use a direct byte buffer to hold the encoded bytes and limit the intermediate on-heap buffer to 64KB only. This optimization kicks in only if the max bytes required by the string being serialized is greater than 64KB. With this patch I can index the same 100MB JSON document with 1200MB of heap. Yonik Seeley - Thoughts?
        Hide
        Mikhail Khludnev added a comment -

        Shalin,

        • couldn't it turn that calling frequent call allocateDirect()&clear() takes too much time? in this case isn't it worth to reuse directBuffer across writeStr() calls as JavaBinCodec's field.
        • I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag, is it a big problem if we loop ByteUtils.UTF16toUTF8() twice, the first time to calculate the length and completely dropping the content, then writing content in the second time loop.
        • just curious, how much efforts does it take to extend javabin format by http-like chunks?
        Show
        Mikhail Khludnev added a comment - Shalin, couldn't it turn that calling frequent call allocateDirect()&clear() takes too much time? in this case isn't it worth to reuse directBuffer across writeStr() calls as JavaBinCodec's field. I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag, is it a big problem if we loop ByteUtils.UTF16toUTF8() twice, the first time to calculate the length and completely dropping the content, then writing content in the second time loop. just curious, how much efforts does it take to extend javabin format by http-like chunks?
        Hide
        Shalin Shekhar Mangar added a comment -

        couldn't it turn that calling frequent call allocateDirect()&clear() takes too much time? in this case isn't it worth to reuse directBuffer across writeStr() calls as JavaBinCodec's field.

        Yes, allocateDirect() can be slower and we should reuse the buffer as much as possible. This was just an idea as a patch. I don't intend to commit it as it is.

        I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag, is it a big problem if we loop ByteUtils.UTF16toUTF8() twice, the first time to calculate the length and completely dropping the content, then writing content in the second time loop.

        Hmm, interesting idea. We could also have a method calcUTF16toUTF8Length which avoids all the bitwise operators and just returns the required length.

        just curious, how much efforts does it take to extend javabin format by http-like chunks?

        It should be possible. We'll need a new chunked type and an upgrade to the JavaBin version. Or we may be able to get away with modifying only the LogCodec in TransactionLog.

        Show
        Shalin Shekhar Mangar added a comment - couldn't it turn that calling frequent call allocateDirect()&clear() takes too much time? in this case isn't it worth to reuse directBuffer across writeStr() calls as JavaBinCodec's field. Yes, allocateDirect() can be slower and we should reuse the buffer as much as possible. This was just an idea as a patch. I don't intend to commit it as it is. I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag, is it a big problem if we loop ByteUtils.UTF16toUTF8() twice, the first time to calculate the length and completely dropping the content, then writing content in the second time loop. Hmm, interesting idea. We could also have a method calcUTF16toUTF8Length which avoids all the bitwise operators and just returns the required length. just curious, how much efforts does it take to extend javabin format by http-like chunks? It should be possible. We'll need a new chunked type and an upgrade to the JavaBin version. Or we may be able to get away with modifying only the LogCodec in TransactionLog.
        Hide
        Yonik Seeley added a comment -

        limit the intermediate on-heap buffer to 64KB only

        I only glanced at it, but it's probably a little too simplistic. You can't cut UTF16 in random places, encode it as UTF8 and get the same bytes because of 2 char code points.

        I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag

        Yeah, the length is really the only reason we need to buffer and copy.

        We should really consider returning to how v1 of the protocol handled things since it had to do no buffering at all since it simply used String.length(). We just need to consider how to handle back compat of course.

        Show
        Yonik Seeley added a comment - limit the intermediate on-heap buffer to 64KB only I only glanced at it, but it's probably a little too simplistic. You can't cut UTF16 in random places, encode it as UTF8 and get the same bytes because of 2 char code points. I've got that buffering is necessary just because we need to calculate length of encoded bytes for starting tag Yeah, the length is really the only reason we need to buffer and copy. We should really consider returning to how v1 of the protocol handled things since it had to do no buffering at all since it simply used String.length(). We just need to consider how to handle back compat of course.
        Hide
        Shalin Shekhar Mangar added a comment -

        I only glanced at it, but it's probably a little too simplistic. You can't cut UTF16 in random places, encode it as UTF8 and get the same bytes because of 2 char code points.

        Duh, of course, I used to know that once and yet...

        We should really consider returning to how v1 of the protocol handled things since it had to do no buffering at all since it simply used String.length(). We just need to consider how to handle back compat of course.

        Until we go there, how about this patch which has a modified UTF16toUTF8 method that writes to ByteBuffer directly using an intermediate scratch array?

        Show
        Shalin Shekhar Mangar added a comment - I only glanced at it, but it's probably a little too simplistic. You can't cut UTF16 in random places, encode it as UTF8 and get the same bytes because of 2 char code points. Duh, of course, I used to know that once and yet... We should really consider returning to how v1 of the protocol handled things since it had to do no buffering at all since it simply used String.length(). We just need to consider how to handle back compat of course. Until we go there, how about this patch which has a modified UTF16toUTF8 method that writes to ByteBuffer directly using an intermediate scratch array?
        Hide
        Yonik Seeley added a comment -

        Making 3 copies really feels heavyweight (the current code makes a single copy in the case of large strings).
        string->scratch, scratch->off-heap-buffer, off-heap-buffer->scratch, scratch->write
        And this doesn't even use less system memory... just less heap memory.

        I wonder how it would do against Mikhail's idea of just calculating the UTF8 length first.
        If we keep with DirectByteBuffer, we should at least eliminate one of the copies by encoding directly to it?

        Show
        Yonik Seeley added a comment - Making 3 copies really feels heavyweight (the current code makes a single copy in the case of large strings). string->scratch, scratch->off-heap-buffer, off-heap-buffer->scratch, scratch->write And this doesn't even use less system memory... just less heap memory. I wonder how it would do against Mikhail's idea of just calculating the UTF8 length first. If we keep with DirectByteBuffer, we should at least eliminate one of the copies by encoding directly to it?
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks Yonik for reviewing.

        I wrote a small JMH benchmark between different approaches at https://github.com/shalinmangar/solr-jmh-tests

        1. JavaBinCodecBenchmark.testDefaultWriteStr measures unmodified JavaBinCodec
        2. JavaBinCodecBenchmark.testDirectBufferWriteStr is the direct buffer code which made 3 copies
        3. JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr is direct buffer code which writes to off-heap buffer without using an intermediate scratch. A 8KB scratch is used to write contents of off-heap buffer
        4. JavaBinCodecBenchmark.testDoublePassWriteStr uses Mikhail's double pass approach. We calculate the length first using a method which avoids all the bitwise operations and then writes the bytes directly to the FastOutputStream

        Here are the results:

        10 MB JSON
        ------------------------------------------------------------------------------------------
        
        java -server -Xmx2048M -Xms2048M -jar target/benchmarks.jar -wi 3 -i 3 -gc true ".*JavaBinCodecBenchmark.*"
        # Run complete. Total time: 00:05:31
        
        Benchmark                                                 Mode  Cnt   Score   Error  Units
        JavaBinCodecBenchmark.testDefaultWriteStr                thrpt   30  47.662 ± 5.691  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr  thrpt   30  43.079 ± 2.693  ops/s
        JavaBinCodecBenchmark.testDirectBufferWriteStr           thrpt   30  28.578 ± 1.290  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteStr             thrpt   30  34.191 ± 1.268  ops/s
        
        100 MB JSON
        ------------------------------------------------------------------------------------------
        
        java -server -Xmx2048M -Xms2048M -jar target/benchmarks.jar -wi 3 -i 3 -gc true ".*JavaBinCodecBenchmark.*"
        
        # Run complete. Total time: 00:06:38
        
        Benchmark                                                 Mode  Cnt  Score   Error  Units
        JavaBinCodecBenchmark.testDefaultWriteStr                thrpt   30  5.304 ± 0.318  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr  thrpt   30  4.890 ± 0.514  ops/s
        JavaBinCodecBenchmark.testDirectBufferWriteStr           thrpt   30  3.274 ± 0.091  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteStr             thrpt   30  2.194 ± 0.295  ops/s
        

        I am not sure how well JMH works with IO benchmarks but it seems that the double pass approach is too slow and using direct buffer without scratch works best.

        Show
        Shalin Shekhar Mangar added a comment - Thanks Yonik for reviewing. I wrote a small JMH benchmark between different approaches at https://github.com/shalinmangar/solr-jmh-tests JavaBinCodecBenchmark.testDefaultWriteStr measures unmodified JavaBinCodec JavaBinCodecBenchmark.testDirectBufferWriteStr is the direct buffer code which made 3 copies JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr is direct buffer code which writes to off-heap buffer without using an intermediate scratch. A 8KB scratch is used to write contents of off-heap buffer JavaBinCodecBenchmark.testDoublePassWriteStr uses Mikhail's double pass approach. We calculate the length first using a method which avoids all the bitwise operations and then writes the bytes directly to the FastOutputStream Here are the results: 10 MB JSON ------------------------------------------------------------------------------------------ java -server -Xmx2048M -Xms2048M -jar target/benchmarks.jar -wi 3 -i 3 -gc true ".*JavaBinCodecBenchmark.*" # Run complete. Total time: 00:05:31 Benchmark Mode Cnt Score Error Units JavaBinCodecBenchmark.testDefaultWriteStr thrpt 30 47.662 ± 5.691 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr thrpt 30 43.079 ± 2.693 ops/s JavaBinCodecBenchmark.testDirectBufferWriteStr thrpt 30 28.578 ± 1.290 ops/s JavaBinCodecBenchmark.testDoublePassWriteStr thrpt 30 34.191 ± 1.268 ops/s 100 MB JSON ------------------------------------------------------------------------------------------ java -server -Xmx2048M -Xms2048M -jar target/benchmarks.jar -wi 3 -i 3 -gc true ".*JavaBinCodecBenchmark.*" # Run complete. Total time: 00:06:38 Benchmark Mode Cnt Score Error Units JavaBinCodecBenchmark.testDefaultWriteStr thrpt 30 5.304 ± 0.318 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr thrpt 30 4.890 ± 0.514 ops/s JavaBinCodecBenchmark.testDirectBufferWriteStr thrpt 30 3.274 ± 0.091 ops/s JavaBinCodecBenchmark.testDoublePassWriteStr thrpt 30 2.194 ± 0.295 ops/s I am not sure how well JMH works with IO benchmarks but it seems that the double pass approach is too slow and using direct buffer without scratch works best.
        Hide
        Shalin Shekhar Mangar added a comment -

        This patch writes to a byte buffer directly.

        Show
        Shalin Shekhar Mangar added a comment - This patch writes to a byte buffer directly.
        Hide
        Mikhail Khludnev added a comment -

        would you mind to share /home/shalin/temp/debug/lucenesolr-569/jimtests/input.14.json
        also?

        Show
        Mikhail Khludnev added a comment - would you mind to share /home/shalin/temp/debug/lucenesolr-569/jimtests/input.14.json also?
        Hide
        Shalin Shekhar Mangar added a comment -

        You can use this bash script to generate the test files: https://gist.github.com/shalinmangar/de8a9af5577ff000e556

        It will download a book from gutenberg website and create two files input14.json (~10MB) and input140.json (~100MB).

        Show
        Shalin Shekhar Mangar added a comment - You can use this bash script to generate the test files: https://gist.github.com/shalinmangar/de8a9af5577ff000e556 It will download a book from gutenberg website and create two files input14.json (~10MB) and input140.json (~100MB).
        Hide
        Noble Paul added a comment -

        This issue is about reducing the memory consumption. Moving the memory used, from heap to off heap is just kicking the can. Let's use the double-pass option, which actually solves the problem. A person who is indexing huge documents is worried about out of memory than shaving of a couple of milliseconds from indexing time

        Show
        Noble Paul added a comment - This issue is about reducing the memory consumption. Moving the memory used, from heap to off heap is just kicking the can. Let's use the double-pass option, which actually solves the problem. A person who is indexing huge documents is worried about out of memory than shaving of a couple of milliseconds from indexing time
        Hide
        Mikhail Khludnev added a comment -

        some off-top:
        i tried to reproduce a gain of allocating direct buffer. I restructured benchmark to call writeStr() in a loop for ten elements. And it shows no significant difference.

        JavaBinCodecBenchmark.testDefaultWriteStr                         thrpt   30  0.042 ± 0.002  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchReuseBuffWriteStr  thrpt   30  0.038 ± 0.002  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr           thrpt   30  0.037 ± 0.001  ops/s
        
        Show
        Mikhail Khludnev added a comment - some off-top: i tried to reproduce a gain of allocating direct buffer. I restructured benchmark to call writeStr() in a loop for ten elements. And it shows no significant difference. JavaBinCodecBenchmark.testDefaultWriteStr thrpt 30 0.042 ± 0.002 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchReuseBuffWriteStr thrpt 30 0.038 ± 0.002 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr thrpt 30 0.037 ± 0.001 ops/s
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks Noble. I am myself coming around to the idea of using the double pass approach. A small performance hit may be better than going out of memory or interfering with the OS page cache in such use-cases. I'll put up a patch with the double pass approach. Does anyone have any opinions on the magic number (currently 64KB) over which the double pass approach should be used?

        Show
        Shalin Shekhar Mangar added a comment - Thanks Noble. I am myself coming around to the idea of using the double pass approach. A small performance hit may be better than going out of memory or interfering with the OS page cache in such use-cases. I'll put up a patch with the double pass approach. Does anyone have any opinions on the magic number (currently 64KB) over which the double pass approach should be used?
        Hide
        Shalin Shekhar Mangar added a comment -

        SOLR-7971-doublepass.patch uses the double pass approach that Mikhail suggested while writing strings needing larger then 64KB buffers.

        Show
        Shalin Shekhar Mangar added a comment - SOLR-7971 -doublepass.patch uses the double pass approach that Mikhail suggested while writing strings needing larger then 64KB buffers.
        Hide
        Noble Paul added a comment -

        another approach at counting size

        Show
        Noble Paul added a comment - another approach at counting size
        Hide
        Shalin Shekhar Mangar added a comment -

        I added Noble's patch to the JMH tests and I see the following results:

        10 MB JSON
        ==========
        Benchmark                                                  Mode  Cnt   Score   Error  Units
        JavaBinCodecBenchmark.testDefaultWriteStr                 thrpt   30  28.846 ± 1.247  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr   thrpt   30  19.113 ± 0.426  ops/s
        JavaBinCodecBenchmark.testDirectBufferWriteStr            thrpt   30  28.081 ± 0.943  ops/s
        JavaBinCodecBenchmark.testDoublePassCountingOutputStream  thrpt   30  16.167 ± 0.145  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteStr              thrpt   30  22.230 ± 0.506  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteWithScratchStr   thrpt   30  24.608 ± 0.246  ops/s
        
        100MB JSON
        ===========
        Benchmark                                                  Mode  Cnt  Score   Error  Units
        JavaBinCodecBenchmark.testDefaultWriteStr                 thrpt   30  2.338 ± 0.163  ops/s
        JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr   thrpt   30  1.762 ± 0.088  ops/s
        JavaBinCodecBenchmark.testDirectBufferWriteStr            thrpt   30  2.934 ± 0.161  ops/s
        JavaBinCodecBenchmark.testDoublePassCountingOutputStream  thrpt   30  1.613 ± 0.036  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteStr              thrpt   30  1.510 ± 0.186  ops/s
        JavaBinCodecBenchmark.testDoublePassWriteWithScratchStr   thrpt   30  2.424 ± 0.079  ops/s
        

        The CountingNullOutputStream approach is consistently slower than others. Instead of writing directly to the output stream, using an intermediate scratch array is much faster.

        Show
        Shalin Shekhar Mangar added a comment - I added Noble's patch to the JMH tests and I see the following results: 10 MB JSON ========== Benchmark Mode Cnt Score Error Units JavaBinCodecBenchmark.testDefaultWriteStr thrpt 30 28.846 ± 1.247 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr thrpt 30 19.113 ± 0.426 ops/s JavaBinCodecBenchmark.testDirectBufferWriteStr thrpt 30 28.081 ± 0.943 ops/s JavaBinCodecBenchmark.testDoublePassCountingOutputStream thrpt 30 16.167 ± 0.145 ops/s JavaBinCodecBenchmark.testDoublePassWriteStr thrpt 30 22.230 ± 0.506 ops/s JavaBinCodecBenchmark.testDoublePassWriteWithScratchStr thrpt 30 24.608 ± 0.246 ops/s 100MB JSON =========== Benchmark Mode Cnt Score Error Units JavaBinCodecBenchmark.testDefaultWriteStr thrpt 30 2.338 ± 0.163 ops/s JavaBinCodecBenchmark.testDirectBufferNoScratchWriteStr thrpt 30 1.762 ± 0.088 ops/s JavaBinCodecBenchmark.testDirectBufferWriteStr thrpt 30 2.934 ± 0.161 ops/s JavaBinCodecBenchmark.testDoublePassCountingOutputStream thrpt 30 1.613 ± 0.036 ops/s JavaBinCodecBenchmark.testDoublePassWriteStr thrpt 30 1.510 ± 0.186 ops/s JavaBinCodecBenchmark.testDoublePassWriteWithScratchStr thrpt 30 2.424 ± 0.079 ops/s The CountingNullOutputStream approach is consistently slower than others. Instead of writing directly to the output stream, using an intermediate scratch array is much faster.
        Hide
        Shalin Shekhar Mangar added a comment -

        Here's the double pass with scratch approach that appears to be the fastest. I'll commit this shortly.

        Show
        Shalin Shekhar Mangar added a comment - Here's the double pass with scratch approach that appears to be the fastest. I'll commit this shortly.
        Hide
        Yonik Seeley added a comment -

        Rather than have two checks per loop iteration, I wonder if it would be faster to make an inner loop?

        +      if (upto > scratch.length - 4)  {
        +        // a code point may take upto 4 bytes and we don't have enough space, so reset
        +        totalBytes += upto;
        +        fos.write(scratch, 0, upto);
        +        upto = 0;
        +      }
        

        Prob not a big deal though, it's going to be a predictable branch.

        Show
        Yonik Seeley added a comment - Rather than have two checks per loop iteration, I wonder if it would be faster to make an inner loop? + if (upto > scratch.length - 4) { + // a code point may take upto 4 bytes and we don't have enough space, so reset + totalBytes += upto; + fos.write(scratch, 0, upto); + upto = 0; + } Prob not a big deal though, it's going to be a predictable branch.
        Hide
        Shalin Shekhar Mangar added a comment -

        Yeah, I think it should be okay? I don't see how to write this logic as an inner loop without making it more complex.

        Show
        Shalin Shekhar Mangar added a comment - Yeah, I think it should be okay? I don't see how to write this logic as an inner loop without making it more complex.
        Hide
        ASF subversion and git services added a comment -

        Commit 1701115 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1701115 ]

        SOLR-7971: JavaBinCodec now uses a double pass approach to write strings larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size

        Show
        ASF subversion and git services added a comment - Commit 1701115 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1701115 ] SOLR-7971 : JavaBinCodec now uses a double pass approach to write strings larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size
        Hide
        ASF subversion and git services added a comment -

        Commit 1701136 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1701136 ]

        SOLR-7971: JavaBinCodec now uses a double pass approach to write strings larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size

        Show
        ASF subversion and git services added a comment - Commit 1701136 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1701136 ] SOLR-7971 : JavaBinCodec now uses a double pass approach to write strings larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size
        Hide
        Shalin Shekhar Mangar added a comment -

        Thanks everyone!

        Show
        Shalin Shekhar Mangar added a comment - Thanks everyone!
        Hide
        ASF subversion and git services added a comment -

        Commit 1701137 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1701137 ]

        SOLR-7971: Mention output stream in javadoc instead of byte buffer

        Show
        ASF subversion and git services added a comment - Commit 1701137 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1701137 ] SOLR-7971 : Mention output stream in javadoc instead of byte buffer
        Hide
        ASF subversion and git services added a comment -

        Commit 1701138 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1701138 ]

        SOLR-7971: Mention output stream in javadoc instead of byte buffer

        Show
        ASF subversion and git services added a comment - Commit 1701138 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1701138 ] SOLR-7971 : Mention output stream in javadoc instead of byte buffer

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            Shalin Shekhar Mangar
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development