Lucene - Core
  1. Lucene - Core
  2. LUCENE-3939

ClassCastException thrown in the map(String,int,TermVectorOffsetInfo[],int[]) method in org.apache.lucene.index.SortedTermVectorMapper

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.2, 3.1, 3.4, 3.5
    • Fix Version/s: None
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The method "map" in the "SortedTermVectorMapper" class does not check the parameter "term" for the valid values. It throws ClassCastException when called with a invalid string for the parameter "term" (i.e., "var3.map("*", (-1), null, null)"). The exception thrown is due to an explict cast(i.e., casting the return value of termToTVE.get(term) to type "TermVectorEntry").

      Suggested Fixes: Replace the beginning of the method body for the class "SortedTermVectorMapper" by changing it like this:

      public void map(String term, int frequency, TermVectorOffsetInfo[] offsets, int[] positions) {
      if(termToTVE.get(term) instanceof TermVectorEntry)

      { TermVectorEntry entry = (TermVectorEntry) termToTVE.get(term); ... }

      }

        Activity

        Hide
        Michael McCandless added a comment -

        By the way, in 4.0 we've simplified access to term vectors: they use the same API as postings (so you can visit terms/docs/positions), except covering only one document. Term vectors are an inverted index for just one document...

        Show
        Michael McCandless added a comment - By the way, in 4.0 we've simplified access to term vectors: they use the same API as postings (so you can visit terms/docs/positions), except covering only one document. Term vectors are an inverted index for just one document...
        Hide
        Michael McCandless added a comment -

        Hmm, that example has basically the same problem: when you call Collections.reverseOrder() without passing it a Comparator, it will fall back to natural order, ie, try to cast the incoming objects to Comparable... you need to provide your own comparator that directly compares the two TermVectorEntry instances...

        Show
        Michael McCandless added a comment - Hmm, that example has basically the same problem: when you call Collections.reverseOrder() without passing it a Comparator, it will fall back to natural order, ie, try to cast the incoming objects to Comparable... you need to provide your own comparator that directly compares the two TermVectorEntry instances...
        Hide
        SHIN HWEI TAN added a comment -

        Ie, on inserting only one object into it, it does not need to cast that object to Comparator (there's nothing to compare to). But on adding a 2nd object, it will try to cast.

        You are right about this. The ClassCastException is not thrown when there is only one object because there is nothing to compare to.

        The test case below shows that ClassCastException is still thrown even when the Comparator is not null:

         
        public void testMisUse(){
           java.util.Comparator comp = java.util.Collections.reverseOrder();
           org.apache.lucene.index.SortedTermVectorMapper mapper = new org.apache.lucene.index.SortedTermVectorMapper(false, false, comp);
           mapper.setExpectations("", 0, false, false);
           org.apache.lucene.index.TermVectorOffsetInfo[] array1 = new org.apache.lucene.index.TermVectorOffsetInfo[] {};
           mapper.map("", 0, array1, (int[]) null);
           mapper.map("*", 1, array1, (int[]) null);
        }
        
        Show
        SHIN HWEI TAN added a comment - Ie, on inserting only one object into it, it does not need to cast that object to Comparator (there's nothing to compare to). But on adding a 2nd object, it will try to cast. You are right about this. The ClassCastException is not thrown when there is only one object because there is nothing to compare to. The test case below shows that ClassCastException is still thrown even when the Comparator is not null: public void testMisUse(){ java.util.Comparator comp = java.util.Collections.reverseOrder(); org.apache.lucene.index.SortedTermVectorMapper mapper = new org.apache.lucene.index.SortedTermVectorMapper( false , false , comp); mapper.setExpectations("", 0, false , false ); org.apache.lucene.index.TermVectorOffsetInfo[] array1 = new org.apache.lucene.index.TermVectorOffsetInfo[] {}; mapper.map("", 0, array1, ( int []) null ); mapper.map( "*" , 1, array1, ( int []) null ); }
        Hide
        Michael McCandless added a comment -

        For example, if the first invocation of the method "map" is commented out(as below), then there is no exception thrown. In this case, the Comparator is still null.

        This is because of sneakiness/trapiness in TreeSet (and maybe Java's type erasure for generics), I think.

        Ie, on inserting only one object into it, it does not need to cast that object to Comparator (there's nothing to compare to). But on adding a 2nd object, it will try to cast.

        Show
        Michael McCandless added a comment - For example, if the first invocation of the method "map" is commented out(as below), then there is no exception thrown. In this case, the Comparator is still null. This is because of sneakiness/trapiness in TreeSet (and maybe Java's type erasure for generics), I think. Ie, on inserting only one object into it, it does not need to cast that object to Comparator (there's nothing to compare to). But on adding a 2nd object, it will try to cast.
        Hide
        SHIN HWEI TAN added a comment -

        Thanks for the quick response.

        I don't think that passing null as Comparator is the problem. For example, if the first invocation of the method "map" is commented out(as below), then there is no exception thrown. In this case, the Comparator is still null.

        org.apache.lucene.index.SortedTermVectorMapper var3 = new org.apache.lucene.index.SortedTermVectorMapper(false, false,(java.util.Comparator)null);
        var3.setExpectations("", 0, false, false);
        var3.map("*:", (-1), (org.apache.lucene.index.TermVectorOffsetInfo[])null, (int[])null);

        Show
        SHIN HWEI TAN added a comment - Thanks for the quick response. I don't think that passing null as Comparator is the problem. For example, if the first invocation of the method "map" is commented out(as below), then there is no exception thrown. In this case, the Comparator is still null. org.apache.lucene.index.SortedTermVectorMapper var3 = new org.apache.lucene.index.SortedTermVectorMapper(false, false,(java.util.Comparator)null); var3.setExpectations("", 0, false, false); var3.map("*:", (-1), (org.apache.lucene.index.TermVectorOffsetInfo[])null, (int[])null);
        Hide
        Michael McCandless added a comment -

        OK I see this ClassCastException:

            [junit] Testcase: testMisUse(org.apache.lucene.index.TestTermVectorsReader):	Caused an ERROR
            [junit] org.apache.lucene.index.TermVectorEntry cannot be cast to java.lang.Comparable
            [junit] java.lang.ClassCastException: org.apache.lucene.index.TermVectorEntry cannot be cast to java.lang.Comparable
            [junit] 	at java.util.TreeMap.put(TreeMap.java:542)
            [junit] 	at java.util.TreeSet.add(TreeSet.java:238)
            [junit] 	at org.apache.lucene.index.SortedTermVectorMapper.map(SortedTermVectorMapper.java:71)
            [junit] 	at org.apache.lucene.index.TestTermVectorsReader.testMisUse(TestTermVectorsReader.java:456)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
            [junit] 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
            [junit] 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
            [junit] 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
            [junit] 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:630)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:536)
            [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:457)
            [junit] 	at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:508)
            [junit] 	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
            [junit] 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
            [junit] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
            [junit] 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
            [junit] 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
            [junit] 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
            [junit] 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
            [junit] 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
            [junit] 	at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74)
            [junit] 	at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36)
            [junit] 	at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67)
            [junit] 	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
            [junit] 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
            [junit] 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
        

        However, I think it's because you're passing null as the Comparator<TermVectorEntry> to SortedTermVectorMapper, forcing the TreeSet to try to cast the TermVectorEntry to Comparable, which then fails.

        Really you should not pass a null Comparator...

        Show
        Michael McCandless added a comment - OK I see this ClassCastException: [junit] Testcase: testMisUse(org.apache.lucene.index.TestTermVectorsReader): Caused an ERROR [junit] org.apache.lucene.index.TermVectorEntry cannot be cast to java.lang.Comparable [junit] java.lang.ClassCastException: org.apache.lucene.index.TermVectorEntry cannot be cast to java.lang.Comparable [junit] at java.util.TreeMap.put(TreeMap.java:542) [junit] at java.util.TreeSet.add(TreeSet.java:238) [junit] at org.apache.lucene.index.SortedTermVectorMapper.map(SortedTermVectorMapper.java:71) [junit] at org.apache.lucene.index.TestTermVectorsReader.testMisUse(TestTermVectorsReader.java:456) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) [junit] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) [junit] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) [junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:630) [junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:536) [junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67) [junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:457) [junit] at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74) [junit] at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:508) [junit] at org.junit.rules.RunRules.evaluate(RunRules.java:18) [junit] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) [junit] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:146) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) [junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) [junit] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) [junit] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) [junit] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) [junit] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) [junit] at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:74) [junit] at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:36) [junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:67) [junit] at org.junit.rules.RunRules.evaluate(RunRules.java:18) [junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:300) [junit] at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) However, I think it's because you're passing null as the Comparator<TermVectorEntry> to SortedTermVectorMapper, forcing the TreeSet to try to cast the TermVectorEntry to Comparable, which then fails. Really you should not pass a null Comparator...
        Hide
        SHIN HWEI TAN added a comment -

        Thanks for the comment.

        Below is a test case that illustrate the problem: The second invocation to the method "map" throws ClassCastException although it is expected to run normally without any exception.

        org.apache.lucene.index.SortedTermVectorMapper var3 = new org.apache.lucene.index.SortedTermVectorMapper(false, false,(java.util.Comparator)null);
        var3.setExpectations("", 0, false, false);
        org.apache.lucene.index.TermVectorOffsetInfo[] var11 = new org.apache.lucene.index.TermVectorOffsetInfo[] { };
        var3.map("", (-1), var11, (int[])null);
        var3.map("*", (-1), (org.apache.lucene.index.TermVectorOffsetInfo[])null, (int[])null);

        Show
        SHIN HWEI TAN added a comment - Thanks for the comment. Below is a test case that illustrate the problem: The second invocation to the method "map" throws ClassCastException although it is expected to run normally without any exception. org.apache.lucene.index.SortedTermVectorMapper var3 = new org.apache.lucene.index.SortedTermVectorMapper(false, false,(java.util.Comparator)null); var3.setExpectations("", 0, false, false); org.apache.lucene.index.TermVectorOffsetInfo[] var11 = new org.apache.lucene.index.TermVectorOffsetInfo[] { }; var3.map("", (-1), var11, (int[])null); var3.map("*", (-1), (org.apache.lucene.index.TermVectorOffsetInfo[])null, (int[])null);
        Hide
        Michael McCandless added a comment -

        I'm confused on how something's that not a TermVectorEntry can get into the termToTVE map... can you post a small test case showing this problem?

        Show
        Michael McCandless added a comment - I'm confused on how something's that not a TermVectorEntry can get into the termToTVE map... can you post a small test case showing this problem?
        SHIN HWEI TAN created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            SHIN HWEI TAN
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 0.05h
              0.05h
              Remaining:
              Remaining Estimate - 0.05h
              0.05h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development