[PIG-5414] Build failure on Linux ARM64 due to old Apache Avro

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.18.0
• Fix Version/s: 0.18.1
• Component/s: build
• Assignee: Martin Tzvetanov Grigorov
• Reporter: Martin Tzvetanov Grigorov
• Created: 12/Oct/21 11:47
• Updated: 15/Jul/23 20:31

Description

Trying to build Apache Pig on Ubuntu 20.04.3 ARM64 fails because of old version of Snappy and Avro libraries:

 

Testsuite: org.apache.pig.builtin.TestAvroStorage
Tests run: 0, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.1 sec
------------- Standard Output ---------------
2021-10-12 14:43:35,483 [main] INFO  org.apache.pig.impl.util.SpillableMemoryManager  - Selected heap (PS Old Gen) of size 1431830528 to monitor. collectionUsageThreshold = 1064828928, usageThreshold = 1064828928
2021-10-12 14:43:35,489 [main] INFO  org.apache.pig.ExecTypeProvider  - Trying ExecType : LOCAL
2021-10-12 14:43:35,489 [main] INFO  org.apache.pig.ExecTypeProvider  - Picked LOCAL as the ExecType
2021-10-12 14:43:35,515 [main] WARN  org.apache.hadoop.conf.Configuration  - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
2021-10-12 14:43:35,755 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2021-10-12 14:43:35,899 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-10-12 14:43:35,916 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine  - Connecting to hadoop file system at: file:///
2021-10-12 14:43:36,116 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2021-10-12 14:43:36,137 [main] INFO  org.apache.pig.PigServer  - Pig Script ID for the session: PIG-default-01426621-bc19-499f-981e-b13959fe0d84
2021-10-12 14:43:36,137 [main] WARN  org.apache.pig.PigServer  - ATS is disabled since yarn.timeline-service.enabled set to false
2021-10-12 14:43:36,150 [main] INFO  org.apache.pig.builtin.TestAvroStorage  - creating test/org/apache/pig/builtin/avro/data/avro/uncompressed/arraysAsOutputByPig.avro
2021-10-12 14:43:36,502 [main] INFO  org.apache.pig.builtin.TestAvroStorage  - Could not generate avro file: test/org/apache/pig/builtin/avro/data/avro/uncompressed/arraysAsOutputByPig.avro
java.net.ConnectException: Call From martin/127.0.0.1 to localhost:40073 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
        at org.apache.hadoop.ipc.Client.call(Client.java:1479)
        at org.apache.hadoop.ipc.Client.call(Client.java:1412)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
        at com.sun.proxy.$Proxy13.getBlockLocations(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:255)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...
 

 

Issue Links

• duplicates: PIG-5405 Build and test Apache Pig on ARM64 CPU architecture
• links to: GitHub Pull Request #35