Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-829

replication Compression



    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4
    • Component/s: replication (java)
    • Labels:


      From a discussion on the mailing list solr-user, it would be useful to have an option to compress the files sent between servers for replication purposes.

      Files sent across between indexes can be compressed by a large margin allowing for easier replication between sites.

      ...Noted by Noble Paul

      we will use a gzip on both ends of the pipe . On the slave side you can say <str name="zip">true<str> as an extra option to compress and send data from server

      Other thoughts on issue:

      Do keep in mind that compression is a CPU intensive process so it is a trade off between CPU utilization and network bandwidth. I have see cases where compressing the data before a network transfer ended up being slower than without compression because the cost of compression and un-compression was more than the gain in network transfer.

      Why invent something when compression is standard in HTTP? --wunder


        1. solr-829.patch
          18 kB
          Shalin Shekhar Mangar
        2. solr-829.patch
          7 kB
          Akshay K. Ukey
        3. solr-829.patch
          5 kB
          Noble Paul
        4. solr-829.patch
          3 kB
          Akshay K. Ukey
        5. email discussion.txt
          6 kB
          Simon Collins

          Issue Links



              • Assignee:
                shalin Shalin Shekhar Mangar
                scollins Simon Collins
              • Votes:
                0 Vote for this issue
                1 Start watching this issue


                • Created: