Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-766

ava.lang.OutOfMemoryError: Java heap space

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.2.0, 0.7.0
    • Fix Version/s: None
    • Component/s: impl
    • Labels:
      None
    • Environment:

      Hadoop-0.18.3 (cloudera RPMs).
      mapred.child.java.opts=-Xmx1024m

      Description

      My pig script always fails with the following error:

      Java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Arrays.java:2786)
      at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
      at java.io.DataOutputStream.write(DataOutputStream.java:90)
      at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
      at org.apache.pig.data.DataReaderWriter.writeDatum(DataReaderWriter.java:213)
      at org.apache.pig.data.DefaultTuple.write(DefaultTuple.java:291)
      at org.apache.pig.data.DefaultAbstractBag.write(DefaultAbstractBag.java:233)
      at org.apache.pig.data.DataReaderWriter.writeDatum(DataReaderWriter.java:162)
      at org.apache.pig.data.DefaultTuple.write(DefaultTuple.java:291)
      at org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:83)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
      at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:156)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.spillSingleRecord(MapTask.java:857)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:467)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:101)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:219)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:86)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vzaliva Vadim Zaliva
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: