Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5161

review FSDirectory chunking defaults and test the chunking

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.5, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Today there is a loop in SimpleFS/NIOFS:

      try {
                do {
                  final int readLength;
                  if (total + chunkSize > len) {
                    readLength = len - total;
                  } else {
                    // LUCENE-1566 - work around JVM Bug by breaking very large reads into chunks
                    readLength = chunkSize;
                  }
                  final int i = file.read(b, offset + total, readLength);
                  total += i;
                } while (total < len);
              } catch (OutOfMemoryError e) {
      

      I bet if you look at the clover report its untested, because its fixed at 100MB for 32-bit users and 2GB for 64-bit users (are these defaults even good?!).

      Also if you call the setter on a 64-bit machine to change the size, it just totally ignores it. We should remove that, the setter should always work.

      And we should set it to small values in tests so this loop is actually executed.

        Attachments

        1. LUCENE-5161.patch
          1 kB
          Robert Muir
        2. LUCENE-5161.patch
          2 kB
          Robert Muir
        3. LUCENE-5161.patch
          2 kB
          Robert Muir

        Issue Links

          Activity

            People

            • Assignee:
              rcmuir Robert Muir
              Reporter:
              rcmuir Robert Muir

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment