Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8973

TX-frenzy on Zookeeper when collection is put to use

    Details

      Description

      This is to do with a distributed data-race. Core-creation happens at a time when collection is not yet visible to the node. In this case a fallback code-path is used which de-references collection-state lazily (on demand) as opposed to setting a watch and keeping it cached locally.

      Due to this, as requests towards the core mount, it generates ZK fetch for collection proportionately. On a large solr-cloud cluster, this generates several Gbps of TX traffic on ZK nodes. This affects indexing throughput(which floors) in addition to running ZK node out of network bandwidth.

      On smaller solr-cloud clusters its hard to run into, because probability of this race materializing reduces.

        Attachments

        1. SOLR-8973.patch
          3 kB
          Janmejay Singh
        2. SOLR-8973-ZkStateReader.patch
          6 kB
          Scott Blum
        3. SOLR-8973.patch
          2 kB
          Janmejay Singh
        4. SOLR-8973.patch
          7 kB
          Shalin Shekhar Mangar

          Activity

            People

            • Assignee:
              dragonsinth Scott Blum
              Reporter:
              janmejay Janmejay Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: