[SOLR-4793] Solr Cloud can't upload large config files ( > 1MB) to Zookeeper

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Workaround
• Fix Version/s: 7.4, 8.0
• Assignee: Steven Rowe
• Reporter: Son Nguyen
• Created: 07/May/13 16:44
• Updated: 25/Jul/18 12:05
• Resolved: 20/Apr/18 20:10

Description

Zookeeper set znode size limit to 1MB by default. So we can't start Solr Cloud with some large config files, like synonyms.txt.

Jan Høydahl has a good idea:
"SolrCloud is designed with an assumption that you should be able to upload your whole disk-based conf folder into ZK, and that you should be able to add an empty Solr node to a cluster and it would download all config from ZK. So immediately a splitting strategy automatically handled by ZkSolresourceLoader for large files could be one way forward, i.e. store synonyms.txt as e.g. __001_synonyms.txt __002_synonyms.txt...."

Issue Links

• is related to: SOLR-12257 Remove documentation of ZooKeeper sysprop jute.maxbuffer once the blob store is a viable alternative for storing/loading large configuration files everywhere in Solr
• relates to: SOLR-11049 Solr in cloud mode silently fails uploading a big LTR model
• relates to: LUCENE-2899 Add OpenNLP Analysis capabilities as a module
• relates to: SOLR-12163 Ref Guide: Improve Setting Up an External ZK Ensemble page
• relates to: SOLR-11250 Add new LTR model which loads the model definition from the external resource
• relates to: SOLR-12590 Improve Solr resource loader coverage in the ref guide