[SOLR-12210] Perfomance degradation at High Load

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 7.2
• Component/s: Hadoop Integration
• Assignee: Unassigned
• Reporter: ROMAN NAZARENKO
• Created: 10/Apr/18 17:22
• Updated: 08/Jun/19 15:12
• Resolved: 26/Mar/19 20:29

Description

When hdfsDataFactory is used and the incoming data stream is large, the rate of indexation is degraded. This is not a problem when using a local file system. I think the problem is in cleaning hdfsBlockCache. Because the degradation of the indexing rate occurs precisely when this cache reaches its maximum value. The basis for hdfsBlockCache is caffeine cache, but it is configured to clean only when the maximum size is reached. At the same time, nothing prevents using timeEviction, I suggest testing the indexing speed using this type of cache cleaning.

Issue Links

• duplicates: SOLR-7393 HDFS poor indexing performance