Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-829

replication Compression

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.4
    • replication (java)
    • None

    Description

      From a discussion on the mailing list solr-user, it would be useful to have an option to compress the files sent between servers for replication purposes.

      Files sent across between indexes can be compressed by a large margin allowing for easier replication between sites.

      ...Noted by Noble Paul

      we will use a gzip on both ends of the pipe . On the slave side you can say <str name="zip">true<str> as an extra option to compress and send data from server

      Other thoughts on issue:

      Do keep in mind that compression is a CPU intensive process so it is a trade off between CPU utilization and network bandwidth. I have see cases where compressing the data before a network transfer ended up being slower than without compression because the cost of compression and un-compression was more than the gain in network transfer.

      Why invent something when compression is standard in HTTP? --wunder

      Attachments

        1. solr-829.patch
          18 kB
          Shalin Shekhar Mangar
        2. solr-829.patch
          7 kB
          Akshay K. Ukey
        3. solr-829.patch
          5 kB
          Noble Paul
        4. solr-829.patch
          3 kB
          Akshay K. Ukey
        5. email discussion.txt
          6 kB
          Simon Collins

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shalin Shalin Shekhar Mangar
            scollins Simon Collins
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment