Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Seems like snappy isn't being properly loaded when run on mac. This is the exception from the TestAvroStorage.testArrayWithSnappyCompression test.
13/02/03 13:20:49 INFO mapReduceLayer.PigMapOnly$Map: Aliases being processed per job phase (AliasName[line,offset]): M: in[1,6] C: R: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:315) at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:218) at org.xerial.snappy.Snappy.<clinit>(Snappy.java:42) at org.apache.avro.file.SnappyCodec.compress(SnappyCodec.java:43) at org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:349) at org.apache.avro.file.DataFileWriter.writeBlock(DataFileWriter.java:347) at org.apache.avro.file.DataFileWriter.sync(DataFileWriter.java:359) at org.apache.avro.file.DataFileWriter.flush(DataFileWriter.java:366) at org.apache.avro.file.DataFileWriter.close(DataFileWriter.java:373) at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.close(PigAvroRecordWriter.java:44) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.close(PigOutputFormat.java:149) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860) at java.lang.Runtime.loadLibrary0(Runtime.java:845) at java.lang.System.loadLibrary(System.java:1084) at org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52) ... 19 more 13/02/03 13:20:49 WARN mapred.LocalJobRunner: job_local_0001 org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:227) at org.xerial.snappy.Snappy.<clinit>(Snappy.java:42) at org.apache.avro.file.SnappyCodec.compress(SnappyCodec.java:43) at org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:349) at org.apache.avro.file.DataFileWriter.writeBlock(DataFileWriter.java:347) at org.apache.avro.file.DataFileWriter.sync(DataFileWriter.java:359) at org.apache.avro.file.DataFileWriter.flush(DataFileWriter.java:366) at org.apache.avro.file.DataFileWriter.close(DataFileWriter.java:373) at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.close(PigAvroRecordWriter.java:44) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.close(PigOutputFormat.java:149) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)