Accumulo
  1. Accumulo
  2. ACCUMULO-354

Create combiners for wikisearch example

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: contrib
    • Labels:

      Description

      The wikisearch example uses aggregators, which are deprecated in 1.4. Since this is a new example in 1.4, they should be combiners.

        Activity

        Hide
        Josh Elser added a comment -

        Tried to run the wikipedia ingest example on a single node Accumulo instance. Got the following error during the first minor compaction:

        java.util.NoSuchElementException
        	java.util.NoSuchElementException
        		at org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.next(TypedValueCombiner.java:94)
        		at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:49)
        		at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:39)
        		at org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:187)
        		at org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:164)
        		at org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:145)
        		at org.apache.accumulo.core.iterators.WrappingIterator.next(WrappingIterator.java:87)
        		at org.apache.accumulo.core.iterators.user.VersioningIterator.skipRowColumn(VersioningIterator.java:103)
        		at org.apache.accumulo.core.iterators.user.VersioningIterator.next(VersioningIterator.java:53)
        		at org.apache.accumulo.server.tabletserver.Compactor.compactLocalityGroup(Compactor.java:276)
        		at org.apache.accumulo.server.tabletserver.Compactor.call(Compactor.java:139)
        		at org.apache.accumulo.server.tabletserver.MinorCompactor.call(MinorCompactor.java:80)
        		at org.apache.accumulo.server.tabletserver.Tablet.minorCompact(Tablet.java:2135)
        		at org.apache.accumulo.server.tabletserver.Tablet.access$4300(Tablet.java:141)
        		at org.apache.accumulo.server.tabletserver.Tablet$MinorCompactionTask.run(Tablet.java:2214)
        		at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        		at cloudtrace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        		at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        		at java.lang.Thread.run(Thread.java:
        
        MinC failed (null) to create /accumulo/tables/2/default_tablet/F0000001.rf_tmp retrying ...
        	java.util.NoSuchElementException
        		at org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.next(TypedValueCombiner.java:94)
        		at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:49)
        		at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:39)
        		at org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:187)
        		at org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:164)
        		at org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:145)
        		at org.apache.accumulo.core.iterators.WrappingIterator.next(WrappingIterator.java:87)
        		at org.apache.accumulo.core.iterators.user.VersioningIterator.skipRowColumn(VersioningIterator.java:103)
        		at org.apache.accumulo.core.iterators.user.VersioningIterator.next(VersioningIterator.java:53)
        		at org.apache.accumulo.server.tabletserver.Compactor.compactLocalityGroup(Compactor.java:276)
        		at org.apache.accumulo.server.tabletserver.Compactor.call(Compactor.java:139)
        		at org.apache.accumulo.server.tabletserver.MinorCompactor.call(MinorCompactor.java:80)
        		at org.apache.accumulo.server.tabletserver.Tablet.minorCompact(Tablet.java:2135)
        		at org.apache.accumulo.server.tabletserver.Tablet.access$4300(Tablet.java:141)
        		at org.apache.accumulo.server.tabletserver.Tablet$MinorCompactionTask.run(Tablet.java:2214)
        		at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        		at cloudtrace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        		at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        		at java.lang.Thread.run(Thread.java:662)
        

        Also, I'm running the current 1.4 branch. It's not readily apparent to me what's happening based on the stack trace alone, but I'll take a look at this tomorrow night if you don't figure it out before then, Billie.

        Show
        Josh Elser added a comment - Tried to run the wikipedia ingest example on a single node Accumulo instance. Got the following error during the first minor compaction: java.util.NoSuchElementException java.util.NoSuchElementException at org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.next(TypedValueCombiner.java:94) at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:49) at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:39) at org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:187) at org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:164) at org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:145) at org.apache.accumulo.core.iterators.WrappingIterator.next(WrappingIterator.java:87) at org.apache.accumulo.core.iterators.user.VersioningIterator.skipRowColumn(VersioningIterator.java:103) at org.apache.accumulo.core.iterators.user.VersioningIterator.next(VersioningIterator.java:53) at org.apache.accumulo.server.tabletserver.Compactor.compactLocalityGroup(Compactor.java:276) at org.apache.accumulo.server.tabletserver.Compactor.call(Compactor.java:139) at org.apache.accumulo.server.tabletserver.MinorCompactor.call(MinorCompactor.java:80) at org.apache.accumulo.server.tabletserver.Tablet.minorCompact(Tablet.java:2135) at org.apache.accumulo.server.tabletserver.Tablet.access$4300(Tablet.java:141) at org.apache.accumulo.server.tabletserver.Tablet$MinorCompactionTask.run(Tablet.java:2214) at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) at cloudtrace.instrument.TraceRunnable.run(TraceRunnable.java:47) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java: MinC failed ( null ) to create /accumulo/tables/2/default_tablet/F0000001.rf_tmp retrying ... java.util.NoSuchElementException at org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.next(TypedValueCombiner.java:94) at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:49) at org.apache.accumulo.examples.wikisearch.iterator.TextIndexCombiner.typedReduce(TextIndexCombiner.java:39) at org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:187) at org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:164) at org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:145) at org.apache.accumulo.core.iterators.WrappingIterator.next(WrappingIterator.java:87) at org.apache.accumulo.core.iterators.user.VersioningIterator.skipRowColumn(VersioningIterator.java:103) at org.apache.accumulo.core.iterators.user.VersioningIterator.next(VersioningIterator.java:53) at org.apache.accumulo.server.tabletserver.Compactor.compactLocalityGroup(Compactor.java:276) at org.apache.accumulo.server.tabletserver.Compactor.call(Compactor.java:139) at org.apache.accumulo.server.tabletserver.MinorCompactor.call(MinorCompactor.java:80) at org.apache.accumulo.server.tabletserver.Tablet.minorCompact(Tablet.java:2135) at org.apache.accumulo.server.tabletserver.Tablet.access$4300(Tablet.java:141) at org.apache.accumulo.server.tabletserver.Tablet$MinorCompactionTask.run(Tablet.java:2214) at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) at cloudtrace.instrument.TraceRunnable.run(TraceRunnable.java:47) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java:662) Also, I'm running the current 1.4 branch. It's not readily apparent to me what's happening based on the stack trace alone, but I'll take a look at this tomorrow night if you don't figure it out before then, Billie.

          People

          • Assignee:
            Billie Rinaldi
            Reporter:
            Billie Rinaldi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development