Uploaded image for project: 'Groovy'
  1. Groovy
  2. GROOVY-4851

NullPointerException on iterator() call

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.10, 1.8-rc-3, 1.8-rc-4, 1.8.0
    • Fix Version/s: 2.5.0-beta-1
    • Component/s: groovy-runtime
    • Labels:
      None
    • Environment:
      Linux Mint 10
    • Flags:
      Patch

      Description

      I got with the following code which use http://sourceforge.net/projects/picard/files/picard-tools/

      import net.sf.picard.util.Interval 
      import net.sf.picard.util.IntervalList 
      import net.sf.picard.util.SamLocusIterator 
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset 
      import net.sf.samtools.SAMFileHeader 
      import net.sf.samtools.SAMFileReader 
      import net.sf.samtools.SAMFileReader.ValidationStringency 
      
      bamFile = new File("/media/trx/workspace/Picard/test/ex1.bam") 
      baiFile = new File(bamFile.getAbsolutePath() + ".bai") 
      
      sam = new SAMFileReader(bamFile, baiFile, true) 
      
      sam.setValidationStringency(ValidationStringency.SILENT) 
      if(!sam.hasIndex()) throw new Exception("Missing index") 
      
      SAMFileHeader header = sam.getFileHeader() 
      
      sequences = header.getSequenceDictionary().getSequences() 
      
      for ( i in sequences) { 
              println("${i.getSequenceName()} = ${i.getSequenceLength()}") 
      } 
      
      IntervalList il = new IntervalList(header) 
      il.add(new Interval("chr1",0, 157)) 
      final SamLocusIterator sli = new SamLocusIterator(sam, il, true) 
      sli.setEmitUncoveredLoci(false) 
      
      sli.each { li -> 
              println li.getSequenceName() 
      } 
      

      I got the following error:

      chr1 = 1575 
      chr2 = 1584 
      WARNING	2011-04-30 12:10:29	SamLocusIterator	SamLocusIterator constructed with samReader that has SortOrder == unsorted.  Assuming SAM is coordinate sorted, but exceptions may occur if it is not. 
      Exception in thread "main" java.lang.NullPointerException 
              at net.sf.picard.util.SamLocusIterator.samHasMore(SamLocusIterator.java:223) 
              at net.sf.picard.util.SamLocusIterator.hasNext(SamLocusIterator.java:231) 
              at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1219) 
              at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1196) 
              at org.codehaus.groovy.runtime.dgm$110.invoke(Unknown Source) 
              at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:270) 
              at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52) 
              at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124) 
              at pileup.run(pileup.groovy:33) 
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
              at java.lang.reflect.Method.invoke(Method.java:616) 
              at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) 
              at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) 
              at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058) 
              at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886) 
              at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:793) 
              at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:776) 
              at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:394) 
              at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) 
              at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) 
              at pileup.main(pileup.groovy) 
      

      The following code is written completely in Java and uses the same input file without any error:

      import java.io.File; 
      import java.util.List; 
      import net.sf.picard.util.Interval; 
      import net.sf.picard.util.IntervalList; 
      import net.sf.picard.util.SamLocusIterator; 
      import net.sf.picard.util.SamLocusIterator.LocusInfo; 
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset; 
      import net.sf.samtools.SAMFileHeader; 
      import net.sf.samtools.SAMFileReader; 
      import net.sf.samtools.SAMRecord; 
      import net.sf.samtools.SAMRecordIterator; 
      import net.sf.samtools.SAMSequenceRecord; 
      import net.sf.samtools.SAMFileReader.ValidationStringency; 
      
      
      public class ReadReferenceName { 
      
              /** 
               * @param args 
               * @throws Exception 
               */ 
              public static void main(String[] args) throws Exception { 
                      // TODO Auto-generated method stub 
                      
                      File bamFile = new File("/media/trx/workspace/Picard/test/ex1.bam"); 
                      File baiFile = new File(bamFile.getAbsolutePath() + ".bai");	
                      
                      SAMFileReader sam = new SAMFileReader(bamFile, baiFile, true); 
              sam.setValidationStringency(ValidationStringency.SILENT); 
                      if(!sam.hasIndex()) throw new Exception("Missing index"); 
                      
                      SAMFileHeader header = sam.getFileHeader(); 
                      List<SAMSequenceRecord> sequences = header.getSequenceDictionary().getSequences(); 
                      
                      for(SAMSequenceRecord rec: sequences) { 
                              System.out.println(rec.getSequenceName() + " = " + 
                                              rec.getSequenceLength()); 
                      } 
                      
                      IntervalList il = new IntervalList(header); 
                      il.add(new Interval("chr1",0, 157)); 
              final SamLocusIterator sli = new SamLocusIterator(sam, il, true); 
              sli.setEmitUncoveredLoci(false); 
            
              for (final SamLocusIterator.LocusInfo li : sli) { 
              String sequenceName = li.getSequenceName(); 
              int pos = li.getPosition(); 
              int coverage = li.getRecordAndPositions().size(); 
              System.out.println(sequenceName + " coverage at base " + pos + " = " 
              + coverage); 
              for (int i = 0; i< coverage; i++){ 
              RecordAndOffset rec = li.getRecordAndPositions().get(i); 
              char base = (char)rec.getReadBase(); 
              String readName = rec.getRecord().getReadName(); 
              System.out.println("  base in read " + readName + " = " + base);	
              } 
              } 
              } 
      } 
      

      and it produced this output:

      chr1 = 1575 
      chr2 = 1584 
      WARNING	2011-04-30 19:30:07	SamLocusIterator	SamLocusIterator constructed with samReader that has SortOrder == unsorted.  Assuming SAM is coordinate sorted, but exceptions may occur if it is not. 
      chr1 coverage at base 100 = 1 
        base in read EAS56_57:6:190:289:82 = A 
      chr1 coverage at base 101 = 1 
      ... 
      

      Tim Yates wrote the following work around for this problem:

      import net.sf.picard.util.*
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset
      import net.sf.samtools.*
      import net.sf.samtools.SAMFileReader.ValidationStringency
      import net.sf.picard.filter.*
      import net.sf.samtools.util.CloseableIterator
      
      bamFile = new File("ex1.bam")
      baiFile = new File("ex1.bam.bai")
      
      sam = new SAMFileReader(bamFile, baiFile, true)
      sam.validationStringency = ValidationStringency.SILENT
      if(!sam.hasIndex()) throw new Exception("Missing index")
      
      SAMFileHeader header = sam.fileHeader
      sequences = header.sequenceDictionary.sequences
      
      sequences.each { i ->
        println "$i.sequenceName = $i.sequenceLength"
      }
      
      SamLocusIterator.metaClass.iterator = {
        if ( delegate.@samIterator != null ) {
          throw new IllegalStateException("Cannot call iterator() more than once on SamLocusIterator");
        }
        CloseableIterator<SAMRecord> tempIterator;
        if (delegate.@intervals != null) {
          tempIterator = new SamRecordIntervalIteratorFactory().makeSamRecordIntervalIterator(delegate.@samReader, delegate.@intervals, delegate.@useIndex);
        } else {
          tempIterator = delegate.@samReader.iterator();
        }
        if (delegate.@samFilters != null) {
          tempIterator = new FilteringIterator(tempIterator, new AggregateFilter(delegate.@samFilters));
        }
        delegate.@samIterator = new PeekableIterator<SAMRecord>( tempIterator )
        delegate 
      }
      
      IntervalList il = new IntervalList( header )
      il.add( new Interval( "chr1", 0, 157 ) )
      SamLocusIterator sli = new SamLocusIterator( sam, il, true )
      sli.emitUncoveredLoci = false
      
      sli.each { li ->
        println "$li.sequenceName coverage at base $li.position = ${li.recordAndPositions.size()}"
        (0..<li.recordAndPositions.size()).each { i ->
          rec = li.recordAndPositions.get( i )
      	println "  base in read $rec.record.readName = $rec.readBase"
        }
      }
      

      More information can be found here ( http://groovy.329449.n5.nabble.com/NullPointerException-problem-td4360426.html )

      1. ex1.bam
        122 kB
        Michal
      2. ex1.bam.bai
        0.2 kB
        Michal
      3. GROOVY-4851.patch
        1.0 kB
        Jochen Theodorou

        Issue Links

          Activity

          Hide
          melix Cédric Champeau added a comment -

          Formatting tags

          Show
          melix Cédric Champeau added a comment - Formatting tags
          Hide
          blackdrag Jochen Theodorou added a comment -

          After further investigation I found this minimal example to reproduce the problem:

          class X implements Iterable, Iterator {
              Iterator iterator() {null}
              boolean hasNext(){true}
              def next(){}
              void remove(){}
          }
          
          
          def x = new X()
          assert x.iterator() == null
          

          the call to iterator() will not call the declared method, instead it will call the our Iterator#iterator method in DGM. If you remove the Iterator interface it works just fine, so it is only reproducable with both interfaces present

          Show
          blackdrag Jochen Theodorou added a comment - After further investigation I found this minimal example to reproduce the problem: class X implements Iterable, Iterator { Iterator iterator() { null } boolean hasNext(){ true } def next(){} void remove(){} } def x = new X() assert x.iterator() == null the call to iterator() will not call the declared method, instead it will call the our Iterator#iterator method in DGM. If you remove the Iterator interface it works just fine, so it is only reproducable with both interfaces present
          Hide
          blackdrag Jochen Theodorou added a comment -

          reducing the level of this bug, since it is not a common problem and depends on the special occasion of an Itarable being also an Iterator

          Show
          blackdrag Jochen Theodorou added a comment - reducing the level of this bug, since it is not a common problem and depends on the special occasion of an Itarable being also an Iterator
          Hide
          blackdrag Jochen Theodorou added a comment -

          it must be something in generating the method list for this class, that makes things go wrong. DGM has the ability to shadow methods of the class, but here it is from an interface and the class implementation should take precedence

          Show
          blackdrag Jochen Theodorou added a comment - it must be something in generating the method list for this class, that makes things go wrong. DGM has the ability to shadow methods of the class, but here it is from an interface and the class implementation should take precedence
          Hide
          paulk Paul King added a comment -

          tagging for review for 2.1.0-rc-1 otherwise we should reassign fix version

          Show
          paulk Paul King added a comment - tagging for review for 2.1.0-rc-1 otherwise we should reassign fix version
          Hide
          blackdrag Jochen Theodorou added a comment -

          The attached patch seems to fix the issue, but creates new problems with getMetaClass(). For me it currently looks like a this.getMetaClass() is currently tending to call the DGM version of this, instead of the local getter method. The fix changes this, but that of course breaks some things here and there (35 meta class related failures)

          Due to those failures I think it is not appropriate to have this in Groovy 2.x and the fix has to wait for Groovy 3 instead

          Show
          blackdrag Jochen Theodorou added a comment - The attached patch seems to fix the issue, but creates new problems with getMetaClass(). For me it currently looks like a this.getMetaClass() is currently tending to call the DGM version of this, instead of the local getter method. The fix changes this, but that of course breaks some things here and there (35 meta class related failures) Due to those failures I think it is not appropriate to have this in Groovy 2.x and the fix has to wait for Groovy 3 instead
          Hide
          blackdrag Jochen Theodorou added a comment -

          Setting new fix version to 3.0

          Show
          blackdrag Jochen Theodorou added a comment - Setting new fix version to 3.0
          Hide
          jkemnade Jochen Kemnade added a comment -

          This is fixed in 2.5.0-alpha-1 via GROOVY-7937.

          Show
          jkemnade Jochen Kemnade added a comment - This is fixed in 2.5.0-alpha-1 via GROOVY-7937 .
          Hide
          jwagenleitner John Wagenleitner added a comment -

          Thanks Michal for reporting the issue and thanks to Jochen K. for confirming the fix.

          Show
          jwagenleitner John Wagenleitner added a comment - Thanks Michal for reporting the issue and thanks to Jochen K. for confirming the fix.

            People

            • Assignee:
              jwagenleitner John Wagenleitner
              Reporter:
              trx Michal
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development