[BIGTOP-3572] Oozie smoke test fails on the ARM platform - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Reopened
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: tests
Labels:
None

Description

https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-10-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-9-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-18.04-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-20.04-arm64-deploy/console

It looks like to be caused by the version mismatch of Guava, but I'm not sure why it happens only on arm64 (and maybe ppc64le?) platform.

+ oozie-setup sharelib create -fs hdfs://28c610858918.bigtop.apache.org:8020/
  setting OOZIE_DATA=/var/lib/oozie
  setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
  setting CATALINA_TMPDIR=/var/lib/oozie
  setting CATALINA_PID=/var/run/oozie/oozie.pid
  setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
  setting OOZIE_CONFIG=/etc/oozie/conf
  setting OOZIE_LOG=/var/log/oozie
  setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
  setting OOZIE_DATA=/var/lib/oozie
  setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
  setting CATALINA_TMPDIR=/var/lib/oozie
  setting CATALINA_PID=/var/run/oozie/oozie.pid
  setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
  setting OOZIE_CONFIG=/etc/oozie/conf
  setting OOZIE_LOG=/var/log/oozie
  setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
	at org.apache.oozie.service.XLogService.extractInfoForLogWebService(XLogService.java:218)
	at org.apache.oozie.service.XLogService.init(XLogService.java:203)
	at org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
	at org.apache.oozie.service.Services.<init>(Services.java:111)
	at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:198)
	at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:103)

I also tried to reproduce it, but I couldn't on the m6g EC2 instance.

ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ cat /etc/issue
Ubuntu 18.04.5 LTS \n \l

ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ arch
aarch64
ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -d -C config_ubuntu-18.04.yaml -G -L -k hdfs,yarn,mapreduce,hive,oozie -s oozie -c 1

...

+ oozie-setup sharelib create -fs hdfs://3349cb325787.bigtop.apache.org:8020/
  setting OOZIE_DATA=/var/lib/oozie
  setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
  setting CATALINA_TMPDIR=/var/lib/oozie
  setting CATALINA_PID=/var/run/oozie/oozie.pid
  setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
  setting OOZIE_CONFIG=/etc/oozie/conf
  setting OOZIE_LOG=/var/log/oozie
  setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
  setting OOZIE_DATA=/var/lib/oozie
  setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
  setting CATALINA_TMPDIR=/var/lib/oozie
  setting CATALINA_PID=/var/run/oozie/oozie.pid
  setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
  setting OOZIE_CONFIG=/etc/oozie/conf
  setting OOZIE_LOG=/var/log/oozie
  setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
8324 [main] INFO org.apache.oozie.service.XLogService - 
 ******************************************************************************* 
  STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on [2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable -DgenerateDocs]
  STARTUP MSG:       revision [branch-5.2@8f0e5ee09]@[git@github.com:apache/oozie.git]
*******************************************************************************
8325 [main] INFO oozieops - 
 ******************************************************************************* 
  STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on [2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable -DgenerateDocs]
  STARTUP MSG:       revision [branch-5.2@8f0e5ee09]@[git@github.com:apache/oozie.git]
*******************************************************************************
8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration file [oozie-log4j.properties]
8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration file loaded from [/etc/oozie/conf]
8326 [main] INFO org.apache.oozie.service.XLogService - Log4j reload interval [10 sec]
8333 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie home dir  [/usr/lib/oozie]
8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf dir  [/etc/oozie/conf]
8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf file [oozie-site.xml]
8613 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [use.system.libpath.for.mapreduce.and.pig.jobs] 
8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [oozie.authentication.signature.secret] 
8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [oozie.service.AuthorizationService.security.enabled] 
8626 [main] WARN org.apache.oozie.util.DateUtils - GMT, UTC or Region/City Timezone formats are preferred instead of UTC
8627 [main] WARN org.apache.oozie.service.Services - System ID [oozie-oozi] exceeds maximum length [10], trimming
8629 [main] INFO org.apache.oozie.service.Services - Exiting null Entering NORMAL
8629 [main] INFO oozieops - Exiting null Entering NORMAL
8630 [main] INFO org.apache.oozie.service.Services - Initialized runtime directory [/tmp/oozie-oozi2045196064457886659.dir]
8630 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService, org.apache.oozie.service.HadoopAccessorService]
8630 [main] INFO oozieops - Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService, org.apache.oozie.service.HadoopAccessorService]
8631 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic configuration change, property[oozie.services.ext]=[]
8631 [main] INFO oozieops - Programmatic configuration change, property[oozie.services.ext]=[]
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - JOB_TRACKER_WHITELIST :[], Total entries :0
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - NAME_NODE_WHITELIST :[], Total entries :0
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - Oozie Kerberos Authentication [disabled]
8872 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
9041 [main] INFO org.apache.hadoop.security.SecurityUtil - Updating Configuration
9041 [main] INFO org.apache.oozie.service.Services - Initialized
9044 [main] INFO org.apache.oozie.service.Services - Running with JARs for Hadoop version [3.2.2]
9048 [main] INFO org.apache.oozie.service.Services - Oozie System ID [oozie-oozi] started!
Found Hadoop that supports Erasure Coding. Trying to disable Erasure Coding for path: /user/oozie/share/lib
Done
the destination path for sharelib is: /user/oozie/share/lib/lib_20210713230159
44357 [main] INFO org.apache.oozie.service.Services - Shutdown

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Kengo Seki

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 13/Jul/21 23:04

Updated:: 26/Aug/21 07:44