[YARN-7433] java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Invalid
• Target Version/s: 2.7.1
• Assignee: Unassigned
• Reporter: gehaijiang
• Created: 03/Nov/17 00:09
• Updated: 06/Nov/17 16:45
• Resolved: 06/Nov/17 16:45

Description

From centos6.5 upgrade centos7, hadoop version(2.7.1) is compiled on centos6.5 and Support snappy, the copy runs on centos7.

but yarn task error ，yarn task errorlog：
(hadoop Native Libraries Whether to recompile based on centos7???)
Error: java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.
at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:134)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:165)
at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:114)
at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:97)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1856)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1511)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Error: java.io.IOException: Spill failed