Uploaded image for project: 'Lucene.Net'
  1. Lucene.Net
  2. LUCENENET-640

Sequential IndexWriter performance in concurrent environments.

    XMLWordPrintableJSON

    Details

    • Flags:
      Important

      Description

      When creating Lucene.Net indices in parallel, sequential-like performance is experienced. Profiling 8 concurrent IndexWriter instances writing in parallel shows that WeakIdentityMap::IdentityWeakReference::Equals spends most time garbage collecting (94.91%) and TokenStream::AssertFinal (87.09% garbage collecting) in my preliminary tests (see screenshots).

      The WeakIdentityMap implementation uses an IdentityWeakReference as key, which is implemented as a class. By inspection of this class, it is merely a System.Runtime.InteropServices.GCHandle wrapper as can be seen in the mono project, manually wrapping of this struct in a struct rather than a class - will eliminate some of the immense amounts of garbage collection.

        Attachments

        1. AssertFinalBug.jpg
          29 kB
          Mathias Henriksen
        2. IdentityWeakReferenceBug.jpg
          18 kB
          Mathias Henriksen
        3. overviewBug.jpg
          107 kB
          Mathias Henriksen
        4. Program.cs
          2 kB
          Mathias Henriksen

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                matt2843 Mathias Henriksen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m