[CARBONDATA-3233] JVM is getting crashed during dataload while compressing in snappy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.2
• Assignee: Akash R Nilugal
• Reporter: Akash R Nilugal
• Created: 07/Jan/19 13:51
• Updated: 17/Jan/19 09:16
• Resolved: 17/Jan/19 09:16

Description

when huge dataload is done, some times dataload is failed and jvm is crashed during snappy compression

 

Below is the logs:

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.xerial.snappy.SnappyNative.rawCompress(JJJ)J+0
j org.apache.carbondata.core.datastore.compression.SnappyCompressor.rawCompress(JIJ)J+9
j org.apache.carbondata.core.datastore.page.UnsafeFixLengthColumnPage.compress(Lorg/apache/carbondata/core/datastore/compression/Compressor[B+50
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveCodec.encodeAndCompressPage(Lorg/apache/carbondata/core/datastore/page/ColumnPage;Lorg/apache/carbondata/core/datastore/page/ColumnPageValueConverter;Lorg/apache/carbondata/core/datastore/compression/Compressor[B+85
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveDeltaIntegralCodec$1.encodeData(Lorg/apache/carbondata/core/datastore/page/ColumnPage[B+45
j org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoder.encode(Lorg/apache/carbondata/core/datastore/page/ColumnPage;)Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+2
j org.apache.carbondata.processing.store.TablePage.encodeAndCompressMeasures()[Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+54
j org.apache.carbondata.processing.store.TablePage.encode()V+6
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+86
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(Lorg/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar;Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+2
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Void;+8
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Object;+1
j java.util.concurrent.FutureTask.run()V+42
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub

Issue Links

• links to: GitHub Pull Request #3053
• links to: GitHub Pull Request #3053