Solr
  1. Solr
  2. SOLR-5613

Upgrade Apache Commons Codec to version 1.9 in order to improve performance of BeiderMorseFilter

    Details

      Description

      In version 1.9 of commons-codec project, there are a lot of optimizations in the Beider Morse encoder. This is used by the BeiderMorseFilter in Solr.
      Do you think it is possible to upgrade this dependency ?

        Issue Links

          Activity

          Hide
          Shawn Heisey added a comment -

          I upgraded commons-codec to 1.9 on an up-to-date branch_4x checkout and found that all tests (both Lucene and Solr) passed. This was on a linux machine. I wasn't too surprised by this. I think we can accommodate this request easily.

          Just for giggles, I went even further and upgraded all commons.apache.org components to the newest versions I could find via ivy. All tests still passed. This was on a Windows 8 machine. With so many upgrades, I was really surprised it passed.

          Index: lucene/ivy-versions.properties
          ===================================================================
          --- lucene/ivy-versions.properties      (revision 1555313)
          +++ lucene/ivy-versions.properties      (working copy)
          @@ -19,16 +19,16 @@
           /com.ibm.icu/icu4j = 52.1
           /com.spatial4j/spatial4j = 0.3
           /com.sun.jersey/jersey-core = 1.16
          -/commons-beanutils/commons-beanutils = 1.7.0
          +/commons-beanutils/commons-beanutils = 1.9.0
           /commons-cli/commons-cli = 1.2
          -/commons-codec/commons-codec = 1.7
          +/commons-codec/commons-codec = 1.9
           /commons-collections/commons-collections = 3.2.1
          -/commons-configuration/commons-configuration = 1.6
          -/commons-digester/commons-digester = 2.0
          -/commons-fileupload/commons-fileupload = 1.2.1
          -/commons-io/commons-io = 2.1
          +/commons-configuration/commons-configuration = 1.10
          +/commons-digester/commons-digester = 2.1
          +/commons-fileupload/commons-fileupload = 1.3
          +/commons-io/commons-io = 2.4
           /commons-lang/commons-lang = 2.6
          -/commons-logging/commons-logging = 1.1.1
          +/commons-logging/commons-logging = 1.1.3
           /de.l3s.boilerpipe/boilerpipe = 1.1.0
           /dom4j/dom4j = 1.6.1
           /edu.ucar/netcdf = 4.2-min
          

          I'm not advocating that we upgrade all the components at once, but it looks like we can indeed upgrade them all eventually. I only ran the basic tests, so additional tests (nightly, weekly, etc) need to be done.

          Show
          Shawn Heisey added a comment - I upgraded commons-codec to 1.9 on an up-to-date branch_4x checkout and found that all tests (both Lucene and Solr) passed. This was on a linux machine. I wasn't too surprised by this. I think we can accommodate this request easily. Just for giggles, I went even further and upgraded all commons.apache.org components to the newest versions I could find via ivy. All tests still passed. This was on a Windows 8 machine. With so many upgrades, I was really surprised it passed. Index: lucene/ivy-versions.properties =================================================================== --- lucene/ivy-versions.properties (revision 1555313) +++ lucene/ivy-versions.properties (working copy) @@ -19,16 +19,16 @@ /com.ibm.icu/icu4j = 52.1 /com.spatial4j/spatial4j = 0.3 /com.sun.jersey/jersey-core = 1.16 -/commons-beanutils/commons-beanutils = 1.7.0 +/commons-beanutils/commons-beanutils = 1.9.0 /commons-cli/commons-cli = 1.2 -/commons-codec/commons-codec = 1.7 +/commons-codec/commons-codec = 1.9 /commons-collections/commons-collections = 3.2.1 -/commons-configuration/commons-configuration = 1.6 -/commons-digester/commons-digester = 2.0 -/commons-fileupload/commons-fileupload = 1.2.1 -/commons-io/commons-io = 2.1 +/commons-configuration/commons-configuration = 1.10 +/commons-digester/commons-digester = 2.1 +/commons-fileupload/commons-fileupload = 1.3 +/commons-io/commons-io = 2.4 /commons-lang/commons-lang = 2.6 -/commons-logging/commons-logging = 1.1.1 +/commons-logging/commons-logging = 1.1.3 /de.l3s.boilerpipe/boilerpipe = 1.1.0 /dom4j/dom4j = 1.6.1 /edu.ucar/netcdf = 4.2-min I'm not advocating that we upgrade all the components at once, but it looks like we can indeed upgrade them all eventually. I only ran the basic tests, so additional tests (nightly, weekly, etc) need to be done.
          Hide
          ASF subversion and git services added a comment -

          Commit 1571593 from shalin@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1571593 ]

          SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance

          Show
          ASF subversion and git services added a comment - Commit 1571593 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1571593 ] SOLR-5613 : Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance
          Hide
          ASF subversion and git services added a comment -

          Commit 1571594 from shalin@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1571594 ]

          SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance

          Show
          ASF subversion and git services added a comment - Commit 1571594 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1571594 ] SOLR-5613 : Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance
          Hide
          Shalin Shekhar Mangar added a comment -

          Thanks Thomas and Shawn. I've updated the lib in trunk and branch_4x.

          Shawn, I don't think we should do a wholesale upgrade of libs - it might become hard to pinpoint regressions, if any. I'll open issues for the other upgrades.

          Show
          Shalin Shekhar Mangar added a comment - Thanks Thomas and Shawn. I've updated the lib in trunk and branch_4x. Shawn, I don't think we should do a wholesale upgrade of libs - it might become hard to pinpoint regressions, if any. I'll open issues for the other upgrades.
          Hide
          ASF subversion and git services added a comment -

          Commit 1571610 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1571610 ]

          SOLR-5613: Add checksum files

          Show
          ASF subversion and git services added a comment - Commit 1571610 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1571610 ] SOLR-5613 : Add checksum files
          Hide
          ASF subversion and git services added a comment -

          Commit 1571612 from Uwe Schindler in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1571612 ]

          Merged revision(s) 1571610 from lucene/dev/trunk:
          SOLR-5613: Add checksum files

          Show
          ASF subversion and git services added a comment - Commit 1571612 from Uwe Schindler in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1571612 ] Merged revision(s) 1571610 from lucene/dev/trunk: SOLR-5613 : Add checksum files
          Hide
          Shawn Heisey added a comment -

          Shawn, I don't think we should do a wholesale upgrade of libs - it might become hard to pinpoint regressions, if any.

          I completely agree. I just tried it as an exercise. I really expected it to completely blow up and fail dozens of tests.

          Show
          Shawn Heisey added a comment - Shawn, I don't think we should do a wholesale upgrade of libs - it might become hard to pinpoint regressions, if any. I completely agree. I just tried it as an exercise. I really expected it to completely blow up and fail dozens of tests.
          Hide
          Uwe Schindler added a comment -

          Close issue after release of 4.8.0

          Show
          Uwe Schindler added a comment - Close issue after release of 4.8.0

            People

            • Assignee:
              Shalin Shekhar Mangar
              Reporter:
              Thomas Champagne
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development