Solr
  1. Solr
  2. SOLR-6024

StatsComponent does not work for docValues enabled multiValued fields

    Details

      Description

      Harish Agarwal reported this in solr user mailing list : http://search-lucene.com/m/QTPaoTJXV1

      It is east to re-produce with default example solr setup. Following types are added example schema.xml. And exampledocs are indexed.

       <field name="cat" type="string" indexed="true" stored="true" docValues="true" multiValued="true"/>
        <field name="popularity" type="int" indexed="true" stored="false" docValues="true" multiValued="true"/>
      

      When docValues="true" and multiValued="true" are used at the same time, StatsComponent throws :

      ERROR org.apache.solr.core.SolrCore  – org.apache.solr.common.SolrException: Type mismatch: popularity was indexed as SORTED_SET
      	at org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:193)
      	at org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:699)
      	at org.apache.solr.handler.component.SimpleStats.getStatsFields(StatsComponent.java:319)
      	at org.apache.solr.handler.component.SimpleStats.getStatsCounts(StatsComponent.java:290)
      	at org.apache.solr.handler.component.StatsComponent.process(StatsComponent.java:78)
      	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:221)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1964)
      
      1. SOLR-6024.patch
        16 kB
        Vitaliy Zhovtyuk
      2. SOLR-6024.patch
        5 kB
        Vitaliy Zhovtyuk
      3. SOLR-6024-branch_4x.patch
        37 kB
        Tomás Fernández Löbbe
      4. SOLR-6024-trunk.patch
        25 kB
        Vitaliy Zhovtyuk
      5. SOLR-6024-trunk.patch
        18 kB
        Tomás Fernández Löbbe
      6. SOLR-6024-trunk.patch
        16 kB
        Tomás Fernández Löbbe
      7. SOLR-6024-trunk.patch
        15 kB
        Tomás Fernández Löbbe
      8. SOLR-6024-trunk.patch
        10 kB
        Tomás Fernández Löbbe

        Issue Links

          Activity

          Show
          Ahmet Arslan added a comment - Here are the search URLs that produce error message: http://localhost:8983/solr/collection1/select?stats=true&stats.field=popularity http://localhost:8983/solr/collection1/select?stats=true&stats.field=cat
          Hide
          Vitaliy Zhovtyuk added a comment -

          Issue reproduced in solr-4.9.0.
          Issue not reproduced on trunk, attached test in patch don't get exception on query.
          UnInvertedField invokation was removed in rev:
          1595259 5/16/14 6:39 PM rmuir 1594441, 1593789 LUCENE-5666: Add UninvertingReader

          Show
          Vitaliy Zhovtyuk added a comment - Issue reproduced in solr-4.9.0. Issue not reproduced on trunk, attached test in patch don't get exception on query. UnInvertedField invokation was removed in rev: 1595259 5/16/14 6:39 PM rmuir 1594441, 1593789 LUCENE-5666 : Add UninvertingReader
          Hide
          Vitaliy Zhovtyuk added a comment -

          Added patch based for lucene_solr_4_9 branch fixing issue,
          for fields having docValues called org.apache.solr.request.DocValuesStats#getCounts from rev. 1595259 and UnInvertedField in other cases.

          Show
          Vitaliy Zhovtyuk added a comment - Added patch based for lucene_solr_4_9 branch fixing issue, for fields having docValues called org.apache.solr.request.DocValuesStats#getCounts from rev. 1595259 and UnInvertedField in other cases.
          Hide
          Tomás Fernández Löbbe added a comment -

          The added test fails if the codec is Lucene 4.0. Tested this on branch_4x and trunk:

             [junit4] ERROR   0.22s | StatsComponentTest.testFieldStatisticsDocValuesAndMultiValued <<<
             [junit4]    > Throwable #1: org.apache.solr.common.SolrException: java.lang.UnsupportedOperationException: Lucene 4.0 does not support SortedSet docvalues
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D7B0D3D7FE664ABF:A5CE511C61788BCE]:0)
             [junit4]    > 	at org.apache.solr.util.TestHarness.update(TestHarness.java:247)
             [junit4]    > 	at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
             [junit4]    > 	at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
             [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:676)
             [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:655)
             [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:649)
             [junit4]    > 	at org.apache.solr.handler.component.StatsComponentTest.testFieldStatisticsDocValuesAndMultiValued(StatsComponentTest.java:449)
             [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
             [junit4]    > Caused by: java.lang.UnsupportedOperationException: Lucene 4.0 does not support SortedSet docvalues
             [junit4]    > 	at org.apache.lucene.codecs.lucene40.Lucene40DocValuesWriter.addSortedSetField(Lucene40DocValuesWriter.java:540)
             [junit4]    > 	at org.apache.lucene.index.SortedSetDocValuesWriter.flush(SortedSetDocValuesWriter.java:164)
             [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:142)
             [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96)
             [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:442)
             [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510)
             [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:621)
             [junit4]    > 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2793)
             [junit4]    > 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2952)
             [junit4]    > 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2919)
             [junit4]    > 	at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:582)
             [junit4]    > 	at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
             [junit4]    > 	at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
             [junit4]    > 	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1650)
             [junit4]    > 	at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1627)
             [junit4]    > 	at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157)
             [junit4]    > 	at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:267)
             [junit4]    > 	at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
             [junit4]    > 	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:99)
             [junit4]    > 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
             [junit4]    > 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
             [junit4]    > 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1985)
             [junit4]    > 	at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
             [junit4]    > 	at org.apache.solr.util.TestHarness.update(TestHarness.java:243)
             [junit4]    > 	... 46 more
             [junit4]   2> 19528 T11 oas.SolrTestCaseJ4.deleteCore ###deleteCore
          
          
          Show
          Tomás Fernández Löbbe added a comment - The added test fails if the codec is Lucene 4.0. Tested this on branch_4x and trunk: [junit4] ERROR 0.22s | StatsComponentTest.testFieldStatisticsDocValuesAndMultiValued <<< [junit4] > Throwable #1: org.apache.solr.common.SolrException: java.lang.UnsupportedOperationException: Lucene 4.0 does not support SortedSet docvalues [junit4] > at __randomizedtesting.SeedInfo.seed([D7B0D3D7FE664ABF:A5CE511C61788BCE]:0) [junit4] > at org.apache.solr.util.TestHarness.update(TestHarness.java:247) [junit4] > at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282) [junit4] > at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252) [junit4] > at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:676) [junit4] > at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:655) [junit4] > at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:649) [junit4] > at org.apache.solr.handler.component.StatsComponentTest.testFieldStatisticsDocValuesAndMultiValued(StatsComponentTest.java:449) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: java.lang.UnsupportedOperationException: Lucene 4.0 does not support SortedSet docvalues [junit4] > at org.apache.lucene.codecs.lucene40.Lucene40DocValuesWriter.addSortedSetField(Lucene40DocValuesWriter.java:540) [junit4] > at org.apache.lucene.index.SortedSetDocValuesWriter.flush(SortedSetDocValuesWriter.java:164) [junit4] > at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:142) [junit4] > at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) [junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:442) [junit4] > at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:510) [junit4] > at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:621) [junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2793) [junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2952) [junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2919) [junit4] > at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:582) [junit4] > at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95) [junit4] > at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64) [junit4] > at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1650) [junit4] > at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1627) [junit4] > at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157) [junit4] > at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:267) [junit4] > at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174) [junit4] > at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:99) [junit4] > at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) [junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) [junit4] > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1985) [junit4] > at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131) [junit4] > at org.apache.solr.util.TestHarness.update(TestHarness.java:243) [junit4] > ... 46 more [junit4] 2> 19528 T11 oas.SolrTestCaseJ4.deleteCore ###deleteCore
          Hide
          Tomás Fernández Löbbe added a comment - - edited

          Patch for trunk. Adds Vitaliy Zhovtyuk's test and suppress codec Lucene 4.0 in the test.

          Show
          Tomás Fernández Löbbe added a comment - - edited Patch for trunk. Adds Vitaliy Zhovtyuk 's test and suppress codec Lucene 4.0 in the test.
          Hide
          Tomás Fernández Löbbe added a comment -

          Added docValues fields for testing in trunk

          Show
          Tomás Fernández Löbbe added a comment - Added docValues fields for testing in trunk
          Hide
          Tomás Fernández Löbbe added a comment -

          The test I added can fail in trunk when using a codec older than Lucene 4.5 because of missing values.

          Show
          Tomás Fernández Löbbe added a comment - The test I added can fail in trunk when using a codec older than Lucene 4.5 because of missing values.
          Hide
          Tomás Fernández Löbbe added a comment -

          Suppressing codecs older than Lucene 4.5

          Show
          Tomás Fernández Löbbe added a comment - Suppressing codecs older than Lucene 4.5
          Hide
          Hoss Man added a comment -

          ok ... so from an svn management standpoint, it looks like the cleanest thing to do is backport all of the commits to DocValuesStats.java from trunk to the 4x branch (r1594441, r1594445, r1595259, r1600688, r1602997), combined with your modified version of StatsComponent.java to still conditionally use UnInvertedField.

          i'm still not really comfortable calling this bug "fixed" w/o more tests thought – in particular there is already plumbing in StatsComponentTest.doTestMVFieldStatisticsResult for testing that multivalued field stats – we need to add multivalued=true+docValue=true+indexed=false versions of all the Trie types to the test schema and ensure that method works for them.

          (we should also reach down into the schema like testStatsFacetMultivaluedErrorHandling() does to assert the SchemaField objects have all the properties we expect to help ensure no one changes the docValues or indexed setting and invalidates the test)

          and of course: we should commit the tests to trunk and then backport to 4x.

          Show
          Hoss Man added a comment - ok ... so from an svn management standpoint, it looks like the cleanest thing to do is backport all of the commits to DocValuesStats.java from trunk to the 4x branch (r1594441, r1594445, r1595259, r1600688, r1602997), combined with your modified version of StatsComponent.java to still conditionally use UnInvertedField. i'm still not really comfortable calling this bug "fixed" w/o more tests thought – in particular there is already plumbing in StatsComponentTest.doTestMVFieldStatisticsResult for testing that multivalued field stats – we need to add multivalued=true+docValue=true+indexed=false versions of all the Trie types to the test schema and ensure that method works for them. (we should also reach down into the schema like testStatsFacetMultivaluedErrorHandling() does to assert the SchemaField objects have all the properties we expect to help ensure no one changes the docValues or indexed setting and invalidates the test) and of course: we should commit the tests to trunk and then backport to 4x.
          Hide
          Tomás Fernández Löbbe added a comment -

          ok ... so from an svn management standpoint, it looks like the cleanest thing to do is backport all of the commits to DocValuesStats.java from trunk to the 4x branch (r1594441, r1594445, r1595259, r1600688, r1602997), combined with your modified version of StatsComponent.java to still conditionally use UnInvertedField.

          I see, I was going to ask what was the best way to bring the DocValuesStats class, it has been in trunk for long now.

          and of course: we should commit the tests to trunk and then backport to 4x.

          Yes, that was why I was adding the "-trunk" patches.

          Show
          Tomás Fernández Löbbe added a comment - ok ... so from an svn management standpoint, it looks like the cleanest thing to do is backport all of the commits to DocValuesStats.java from trunk to the 4x branch (r1594441, r1594445, r1595259, r1600688, r1602997), combined with your modified version of StatsComponent.java to still conditionally use UnInvertedField. I see, I was going to ask what was the best way to bring the DocValuesStats class, it has been in trunk for long now. and of course: we should commit the tests to trunk and then backport to 4x. Yes, that was why I was adding the "-trunk" patches.
          Hide
          Tomás Fernández Löbbe added a comment -

          we need to add multivalued=true+docValue=true+indexed=false versions

          The "missing" stat won't work with this configuration (on trunk). It's being calculated as:

          DocSet missing = docs.andNot( searcher.getDocSet(new TermRangeQuery(fieldName, null, null, false, false)));
          
          Show
          Tomás Fernández Löbbe added a comment - we need to add multivalued=true+docValue=true+indexed=false versions The "missing" stat won't work with this configuration (on trunk). It's being calculated as: DocSet missing = docs.andNot( searcher.getDocSet( new TermRangeQuery(fieldName, null , null , false , false )));
          Hide
          Hoss Man added a comment -

          Ok – totally ignore my last comment. aparently i had a stale copy of this page open in my browser, and didn't see any of tomas's updates to this issue after Vitaliy's last patch.

          i haven't looked closely yet, but at first glance Tomas's patch looks like most likely adds all the test concerns i had.

          Show
          Hoss Man added a comment - Ok – totally ignore my last comment. aparently i had a stale copy of this page open in my browser, and didn't see any of tomas's updates to this issue after Vitaliy's last patch. i haven't looked closely yet, but at first glance Tomas's patch looks like most likely adds all the test concerns i had.
          Hide
          Hoss Man added a comment -

          Gah! .... I don't know what's up with my browser, because it just happened again (firefox doesn't like tomas!)

          Yes, that was why I was adding the "-trunk" patches.

          Yeah ... i think it makes sense to focus on adding test coverage to trunk (and ferreting out any subtle bugs we've been missing due to the lack of test coverate) and then worry about backporting the individual changes to 4x and manually applying the neccessary StatsComponent.java change to conditionally use UnInvertedField when needed.

          The "missing" stat won't work with this configuration (on trunk). It's being calculated as:

          isn't that a bug then? we should definitely be able to get "missing" info from the DocValues API, and we should use it instead of depending on indexed=true for a docValues field. right?

          Show
          Hoss Man added a comment - Gah! .... I don't know what's up with my browser, because it just happened again (firefox doesn't like tomas!) Yes, that was why I was adding the "-trunk" patches. Yeah ... i think it makes sense to focus on adding test coverage to trunk (and ferreting out any subtle bugs we've been missing due to the lack of test coverate) and then worry about backporting the individual changes to 4x and manually applying the neccessary StatsComponent.java change to conditionally use UnInvertedField when needed. The "missing" stat won't work with this configuration (on trunk). It's being calculated as: isn't that a bug then? we should definitely be able to get "missing" info from the DocValues API, and we should use it instead of depending on indexed=true for a docValues field. right?
          Hide
          Tomás Fernández Löbbe added a comment -

          isn't that a bug then?

          I think it is. I'd tackle that in a different Jira, it has a workaround (use index=true) and it reproduces in trunk (unlike the issue described here).
          After a quick look, I think it would be easy to count the missing values for the complete field, but not trivial to count the missing values when using "stats.facet" (for each different value)

          Show
          Tomás Fernández Löbbe added a comment - isn't that a bug then? I think it is. I'd tackle that in a different Jira, it has a workaround (use index=true) and it reproduces in trunk (unlike the issue described here). After a quick look, I think it would be easy to count the missing values for the complete field, but not trivial to count the missing values when using "stats.facet" (for each different value)
          Hide
          Tomás Fernández Löbbe added a comment -

          I created SOLR-6452 for the bug with the "missing" stat

          Show
          Tomás Fernández Löbbe added a comment - I created SOLR-6452 for the bug with the "missing" stat
          Hide
          Tomás Fernández Löbbe added a comment -

          Added preconditions to the new test.
          Test for docValues=true & indexed=false (commented out for now)

          Show
          Tomás Fernández Löbbe added a comment - Added preconditions to the new test. Test for docValues=true & indexed=false (commented out for now)
          Hide
          Vitaliy Zhovtyuk added a comment -

          Patch based on latest added trunk patch.
          Added stats calculation tests for docValues and multiValued fields of float and integer numeric types, added calculate distinct count, added stats.facet query on docValues field (leads to field type exception)

          Show
          Vitaliy Zhovtyuk added a comment - Patch based on latest added trunk patch. Added stats calculation tests for docValues and multiValued fields of float and integer numeric types, added calculate distinct count, added stats.facet query on docValues field (leads to field type exception)
          Hide
          Hoss Man added a comment -

          tests looks good, precommit looks fine ... i tried to ping tomas earlier asking if he wanted ot commit and deal with the backpot or not but i don't think he saw it so i'm going to go ahead and commit to trunk and look into doing the backport & fix on 4.x

          Show
          Hoss Man added a comment - tests looks good, precommit looks fine ... i tried to ping tomas earlier asking if he wanted ot commit and deal with the backpot or not but i don't think he saw it so i'm going to go ahead and commit to trunk and look into doing the backport & fix on 4.x
          Hide
          ASF subversion and git services added a comment -

          Commit 1622386 from hossman@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1622386 ]

          SOLR-6024: Fix StatsComponent when using docValues=true multiValued=true (bug only on 4x trunk commit is just test changes demonstrating correct behavior)

          Show
          ASF subversion and git services added a comment - Commit 1622386 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1622386 ] SOLR-6024 : Fix StatsComponent when using docValues=true multiValued=true (bug only on 4x trunk commit is just test changes demonstrating correct behavior)
          Hide
          Hoss Man added a comment -

          schema11.xml backport is pretty hairy due to divergence between trunk & 4x (removing deprecated types) and then trunk re-adding fieldTypes with the same name as old types.

          going to leave this alone for now and pick it back up tomorow - tomas said on IRC he might take a look at it first, we may wind up doing a tag team backport.

          Show
          Hoss Man added a comment - schema11.xml backport is pretty hairy due to divergence between trunk & 4x (removing deprecated types) and then trunk re-adding fieldTypes with the same name as old types. going to leave this alone for now and pick it back up tomorow - tomas said on IRC he might take a look at it first, we may wind up doing a tag team backport.
          Hide
          Tomás Fernández Löbbe added a comment -

          This patch includes the last changes in trunk plus the changes needed for branch 4x.
          It also include DocValuesStats (should probably be merged from trunk instead)

          Show
          Tomás Fernández Löbbe added a comment - This patch includes the last changes in trunk plus the changes needed for branch 4x. It also include DocValuesStats (should probably be merged from trunk instead)
          Hide
          Hoss Man added a comment -

          This patch includes the last changes in trunk plus the changes needed for branch 4x.

          It also include DocValuesStats (should probably be merged from trunk instead)

          thanks tomas, looks good to me - currently running precommit (and another round of tests just to be safe) on 4x and then i'll commit.

          FWIW, here's my backporting steps...

          # NOTE: working dir is 4x checkout, ../dev is trunk checkout...
          
          svn merge -c 1622386 ../dev/
          
          # manually massaged schema11.xml, StatsComponent.java per Tomas's 4x patch
          # confirm all tests *except* the new StatsComponentTest pass
          
          # manually copy DocValuesStats.java from Tomas's 4x patch
          # confirm StatsComponentTest passes now
          
          svn add solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          
          svn merge --record-only -c 1594441 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          svn merge --record-only -c 1594445 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          svn merge --record-only -c 1595259 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          svn merge --record-only -c 1600688 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          svn merge --record-only -c 1602997 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          
          Show
          Hoss Man added a comment - This patch includes the last changes in trunk plus the changes needed for branch 4x. It also include DocValuesStats (should probably be merged from trunk instead) thanks tomas, looks good to me - currently running precommit (and another round of tests just to be safe) on 4x and then i'll commit. FWIW, here's my backporting steps... # NOTE: working dir is 4x checkout, ../dev is trunk checkout... svn merge -c 1622386 ../dev/ # manually massaged schema11.xml, StatsComponent.java per Tomas's 4x patch # confirm all tests *except* the new StatsComponentTest pass # manually copy DocValuesStats.java from Tomas's 4x patch # confirm StatsComponentTest passes now svn add solr/core/src/java/org/apache/solr/request/DocValuesStats.java svn merge --record-only -c 1594441 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java svn merge --record-only -c 1594445 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java svn merge --record-only -c 1595259 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java svn merge --record-only -c 1600688 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java svn merge --record-only -c 1602997 ../dev solr/core/src/java/org/apache/solr/request/DocValuesStats.java
          Hide
          ASF subversion and git services added a comment -

          Commit 1622525 from hossman@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1622525 ]

          SOLR-6024: Fix StatsComponent when using docValues=true multiValued=true (merge r1622386, and DocValuesStats.java specific changes from r1594441, r1594445, r1595259, r1600688, r1602997)

          Show
          ASF subversion and git services added a comment - Commit 1622525 from hossman@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1622525 ] SOLR-6024 : Fix StatsComponent when using docValues=true multiValued=true (merge r1622386, and DocValuesStats.java specific changes from r1594441, r1594445, r1595259, r1600688, r1602997)
          Hide
          Hoss Man added a comment -

          I think we're all good.

          Show
          Hoss Man added a comment - I think we're all good.
          Hide
          ASF subversion and git services added a comment -

          Commit 1624097 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_10'
          [ https://svn.apache.org/r1624097 ]

          SOLR-6024: Fix StatsComponent when using docValues=true multiValued=true (merge r1622525)

          Show
          ASF subversion and git services added a comment - Commit 1624097 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1624097 ] SOLR-6024 : Fix StatsComponent when using docValues=true multiValued=true (merge r1622525)
          Hide
          Michael McCandless added a comment -

          Bulk close for Lucene/Solr 4.10.1 release

          Show
          Michael McCandless added a comment - Bulk close for Lucene/Solr 4.10.1 release
          Hide
          Elran Dvir added a comment -

          Hi All,

          I am trying to apply this patch on Solr 4.8.
          I have compilation problems with the class DocValuesStats.
          I get the following errors:
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project solr-core: Compilation failure: Compilation failure:
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[88,21] cannot find symbol
          [ERROR] symbol: method emptySortedSet()
          [ERROR] location: class org.apache.lucene.index.DocValues
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[116,28] cannot find symbol
          [ERROR] symbol: method emptySortedSet()
          [ERROR] location: class org.apache.lucene.index.DocValues
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[128,28] cannot find symbol
          [ERROR] symbol: method emptySorted()
          [ERROR] location: class org.apache.lucene.index.DocValues
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[139,34] method lookupOrd in class org.apache.lucene.index.SortedSetDocValues cannot be applied to given types;
          [ERROR] required: long,org.apache.lucene.util.BytesRef
          [ERROR] found: int
          [ERROR] reason: actual and formal argument lists differ in length
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[165,55] cannot find symbol
          [ERROR] symbol: method getGlobalOrds(int)
          [ERROR] location: variable map of type org.apache.lucene.index.MultiDocValues.OrdinalMap
          [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java:[183,55] cannot find symbol
          [ERROR] symbol: method getGlobalOrds(int)
          [ERROR] location: variable map of type org.apache.lucene.index.MultiDocValues.OrdinalMap

          I guess these methods are implemented in newer versions of Solr/lucene.

          How can I fix it in 4.8?
          What is the oldest version the patch can be applied to?

          Thank you very much.

          Show
          Elran Dvir added a comment - Hi All, I am trying to apply this patch on Solr 4.8. I have compilation problems with the class DocValuesStats. I get the following errors: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project solr-core: Compilation failure: Compilation failure: [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [88,21] cannot find symbol [ERROR] symbol: method emptySortedSet() [ERROR] location: class org.apache.lucene.index.DocValues [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [116,28] cannot find symbol [ERROR] symbol: method emptySortedSet() [ERROR] location: class org.apache.lucene.index.DocValues [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [128,28] cannot find symbol [ERROR] symbol: method emptySorted() [ERROR] location: class org.apache.lucene.index.DocValues [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [139,34] method lookupOrd in class org.apache.lucene.index.SortedSetDocValues cannot be applied to given types; [ERROR] required: long,org.apache.lucene.util.BytesRef [ERROR] found: int [ERROR] reason: actual and formal argument lists differ in length [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [165,55] cannot find symbol [ERROR] symbol: method getGlobalOrds(int) [ERROR] location: variable map of type org.apache.lucene.index.MultiDocValues.OrdinalMap [ERROR] /D:/ckp/src/solr_4.8/solr/core/src/java/org/apache/solr/request/DocValuesStats.java: [183,55] cannot find symbol [ERROR] symbol: method getGlobalOrds(int) [ERROR] location: variable map of type org.apache.lucene.index.MultiDocValues.OrdinalMap I guess these methods are implemented in newer versions of Solr/lucene. How can I fix it in 4.8? What is the oldest version the patch can be applied to? Thank you very much.

            People

            • Assignee:
              Tomás Fernández Löbbe
              Reporter:
              Ahmet Arslan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development