17/08/04 10:51:47 INFO InteractiveSession$: Creating Interactive session 9: [owner: phasanye, request: [kind: spark, proxyUser: Some(phasanye), driverMemory: 4G, executorMemory: 2G, executorCores: 2, numExecutors: 5, queue: spark_default, conf: spark.jars.packages -> com.linkedin.spark-avro:spark-avro_2.10:3.2.0.4, heartbeatTimeoutInSecond: 3600]] 17/08/04 10:51:47 WARN RSCConf: The configuration key livy.rsc.proxy_user has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.rsc.proxy-user instead. 17/08/04 10:51:47 WARN RSCConf: The configuration key livy.rsc.driver_class has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.rsc.driver-class instead. 17/08/04 10:51:47 INFO RpcServer: Connected to the port 10000 17/08/04 10:51:47 WARN RSCConf: Your hostname, example.hostname.com, resolves to a loopback address, but we couldn't find any external IP address! 17/08/04 10:51:47 WARN RSCConf: Set livy.rsc.rpc.server.address if you need to bind to another address. 17/08/04 10:51:47 DEBUG RSCClient: Sending JobRequest[c0a15803-95bf-466d-bdaf-7031849b1afd]. 17/08/04 10:51:47 INFO InteractiveSessionManager: Registering new session 9 17/08/04 10:51:47 INFO ServletLoggerFilter: User: phasanye IP : 172.23.243.116 {"kind": "spark", "proxyUser": "phasanye", "heartbeatTimeoutInSecond": 3600, "driverMemory": "4G", "queue": "spark_default", "numExecutors": 5, "conf": {"spark.jars.packages": "com.linkedin.spark-avro:spark-avro_2.10:3.2.0.4"}, "executorCores": 2, "executorMemory": "2G"}?? 17/08/04 10:51:47 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:51:48 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:51:48 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:51:48 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,:: loading settings :: file = /export/apps/spark/commonconf/ivysettings.xml) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,Ivy Default Cache set to: /export/home/livy/.ivy2/cache) 17/08/04 10:51:50 WARN ContextLauncher: Child process exited with code 1. 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,The jars for the packages stored in: /export/home/livy/.ivy2/jars) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,:: loading settings :: url = jar:file:/export/apps/spark/spark-bin-H2.6.0_21043/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,com.linkedin.spark-avro#spark-avro_2.10 added as a dependency) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: ,:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , confs: [default]) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.linkedin.spark-avro#spark-avro_2.10;3.2.0.4 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.avro#avro;1.7.6 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.avro#avro-mapred;1.7.7 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-client;2.2.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-yarn-api;2.2.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.slf4j#slf4j-api;1.7.5 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.slf4j#slf4j-log4j12;1.7.5 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found log4j#log4j;1.2.17 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-annotations;2.2.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.google.inject.extensions#guice-servlet;3.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.google.inject#guice;3.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found javax.inject#javax.inject;1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found aopalliance#aopalliance;1.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.sonatype.sisu.inject#cglib;2.2.1-v20090111 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found asm#asm;3.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found io.netty#netty;3.6.2.Final in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.google.protobuf#protobuf-java;2.5.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-io#commons-io;2.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.jersey.jersey-test-framework#jersey-test-framework-grizzly2;1.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.jersey#jersey-server;1.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.jersey#jersey-json;1.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.codehaus.jettison#jettison;1.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found stax#stax-api;1.0.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.xml.bind#jaxb-impl;2.2.3-1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found javax.xml.bind#jaxb-api;2.2.2 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found javax.activation#activation;1.1 in public) 17/08/04 10:51:50 ERROR RSCClient: Failed to connect to context. java.io.IOException: Child process exited with code 1. at com.cloudera.livy.rsc.ContextLauncher$ChildProcess$1.run(ContextLauncher.java:383) at com.cloudera.livy.rsc.ContextLauncher$ChildProcess$2.run(ContextLauncher.java:432) at java.lang.Thread.run(Thread.java:745) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-core-asl;1.8.3 in public) 17/08/04 10:51:50 INFO RSCClient: Failing pending job c0a15803-95bf-466d-bdaf-7031849b1afd due to shutdown. 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-mapper-asl;1.8.3 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-jaxrs;1.8.3 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-xc;1.8.3 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.jersey.contribs#jersey-guice;1.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-common;2.2.0 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.google.guava#guava;11.0.2 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.google.code.findbugs#jsr305;1.3.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-cli#commons-cli;1.2 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.apache.commons#commons-math;2.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found xmlenc#xmlenc;0.52 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-httpclient#commons-httpclient;3.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-codec#commons-codec;1.4 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-net#commons-net;3.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found javax.servlet#servlet-api;2.5 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.mortbay.jetty#jetty;6.1.26 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found org.mortbay.jetty#jetty-util;6.1.26 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found com.sun.jersey#jersey-core;1.9 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-logging#commons-logging;1.1.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found net.java.dev.jets3t#jets3t;0.6.1 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-lang#commons-lang;2.5 in public) 17/08/04 10:51:50 INFO LineBufferedStream: (stdout: , found commons-configuration#commons-configuration;1.6 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-collections#commons-collections;3.2.1 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-digester#commons-digester;1.8 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-beanutils#commons-beanutils;1.7.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found xml-apis#xml-apis;1.0.b2 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-beanutils#commons-beanutils-core;1.8.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-core-asl;1.8.8 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-mapper-asl;1.8.8 in public) 17/08/04 10:51:50 DEBUG InteractiveSession: InteractiveSession 9 session state change from starting to error 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found com.thoughtworks.paranamer#paranamer;2.3 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.xerial.snappy#snappy-java;1.0.5 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.commons#commons-compress;1.4.1 in public) 17/08/04 10:51:51 INFO InteractiveSession: Failed to ping RSC driver for session 9. Killing application. 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.tukaani#xz;1.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-auth;2.2.0 in public) 17/08/04 10:51:51 INFO InteractiveSession: Stopping InteractiveSession 9... 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found com.jcraft#jsch;0.1.42 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found tomcat#jasper-compiler;5.5.23 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found tomcat#jasper-runtime;5.5.23 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-el#commons-el;1.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found javax.servlet.jsp#jsp-api;2.1 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-hdfs;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found com.google.guava#guava;14.0.1 in public) 17/08/04 10:51:51 DEBUG InteractiveSession: InteractiveSession 9 session state change from error to shutting_down 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-mapper-asl;1.9.11 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.codehaus.jackson#jackson-core-asl;1.9.11 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-codec#commons-codec;1.5 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found commons-io#commons-io;2.4 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.slf4j#slf4j-api;1.7.10 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.slf4j#slf4j-log4j12;1.7.10 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-mapreduce-client-common;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-yarn-common;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-yarn-client;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-yarn-server-common;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.zookeeper#zookeeper;3.4.5 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found jline#jline;0.9.94 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found junit#junit;3.8.1 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.jboss.netty#netty;3.2.2.Final in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found io.netty#netty;3.8.0.Final in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-yarn-server-nodemanager;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , found org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0 in public) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,:: resolution report :: resolve 1155ms :: artifacts dl 31ms) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , :: evicted modules:) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.slf4j#slf4j-api;1.7.5 by [org.slf4j#slf4j-api;1.7.10] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.slf4j#slf4j-log4j12;1.7.5 by [org.slf4j#slf4j-log4j12;1.7.10] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , io.netty#netty;3.6.2.Final by [io.netty#netty;3.8.0.Final] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-io#commons-io;2.1 by [commons-io#commons-io;2.4] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-core-asl;1.8.3 by [org.codehaus.jackson#jackson-core-asl;1.8.8] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-mapper-asl;1.8.3 by [org.codehaus.jackson#jackson-mapper-asl;1.8.8] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , com.google.guava#guava;11.0.2 by [com.google.guava#guava;14.0.1] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , com.google.code.findbugs#jsr305;1.3.9 transitively in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-codec#commons-codec;1.3 by [commons-codec#commons-codec;1.4] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-logging#commons-logging;1.1 by [commons-logging#commons-logging;1.1.1] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-codec#commons-codec;1.4 by [commons-codec#commons-codec;1.5] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-lang#commons-lang;2.4 by [commons-lang#commons-lang;2.5] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-core-asl;1.8.8 by [org.codehaus.jackson#jackson-core-asl;1.9.11] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-mapper-asl;1.8.8 by [org.codehaus.jackson#jackson-mapper-asl;1.9.11] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.apache.avro#avro;1.7.4 by [org.apache.avro#avro;1.7.6] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-core-asl;1.9.13 by [org.codehaus.jackson#jackson-core-asl;1.8.8] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.codehaus.jackson#jackson-mapper-asl;1.9.13 by [org.codehaus.jackson#jackson-mapper-asl;1.8.8] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , org.slf4j#slf4j-api;1.6.4 by [org.slf4j#slf4j-api;1.7.5] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , javax.servlet#servlet-api;2.4 by [javax.servlet#servlet-api;2.5] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , commons-logging#commons-logging;1.0.3 by [commons-logging#commons-logging;1.1.1] in [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ---------------------------------------------------------------------) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , | | modules || artifacts |) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , | conf | number| search|dwnlded|evicted|| number|dwnlded|) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ---------------------------------------------------------------------) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , | default | 96 | 0 | 0 | 20 || 76 | 0 |) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ---------------------------------------------------------------------) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,:: retrieving :: org.apache.spark#spark-submit-parent) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , confs: [default]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , 0 artifacts copied, 76 already retrieved (0kB/20ms)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,17/08/04 10:51:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,17/08/04 10:51:50 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,17/08/04 10:51:50 INFO client.RMProxy: Connecting to ResourceManager at example.resourcemanager.com/10.150.1.58:8032) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,17/08/04 10:51:50 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,Exception in thread "main" java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "example.hostname.com/10.150.27.47"; destination host is: "example.resourcemanager.com":8032; ) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client.call(Client.java:1473)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client.call(Client.java:1400)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at com.sun.proxy.$Proxy12.getClusterMetrics(Unknown Source)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:208)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at java.lang.reflect.Method.invoke(Method.java:483)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at com.sun.proxy.$Proxy13.getClusterMetrics(Unknown Source)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:467)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:155)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:155)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:59)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:154)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client.run(Client.scala:1154)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.yarn.Client.main(Client.scala)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at java.lang.reflect.Method.invoke(Method.java:483)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:171)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:169)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at java.security.AccessController.doPrivileged(Native Method)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at javax.security.auth.Subject.doAs(Subject.java:422)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:214)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:128)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,Caused by: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:681)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at java.security.AccessController.doPrivileged(Native Method)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at javax.security.auth.Subject.doAs(Subject.java:422)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:644)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:731)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:369)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client.call(Client.java:1439)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ... 34 more) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:554)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:369)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:723)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:719)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at java.security.AccessController.doPrivileged(Native Method)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at javax.security.auth.Subject.doAs(Subject.java:422)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:718)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ... 37 more) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: ,Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)) 17/08/04 10:51:51 INFO LineBufferedStream: (stdout: , ... 46 more) 17/08/04 10:51:51 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:51:54 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/7] user [phasanye] authenticated 17/08/04 10:51:55 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:51:58 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:51:59 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:51:59 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:52:01 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:04 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:06 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/7] user [phasanye] authenticated 17/08/04 10:52:08 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:10 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:52:11 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:52:11 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:15 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:17 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/7] user [phasanye] authenticated 17/08/04 10:52:19 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:19 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:21 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:52:22 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:22 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:52:25 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:28 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/7] user [phasanye] authenticated 17/08/04 10:52:28 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:32 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:32 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:52:33 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:52:35 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:38 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:39 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/7] user [phasanye] authenticated 17/08/04 10:52:41 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:44 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/8] user [phasanye] authenticated 17/08/04 10:52:44 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/6] user [phasanye] authenticated 17/08/04 10:52:44 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:48 DEBUG AuthenticationFilter: Request [https://example.com:8998/sessions/9] user [phasanye] authenticated 17/08/04 10:52:51 WARN SparkYarnApp: Deleting a session while its YARN application is not found. 17/08/04 10:52:51 ERROR SparkYarnApp: Error whiling refreshing YARN state: java.lang.Exception: spark-submit exited with code 1}. :: loading settings :: file = /export/apps/spark/commonconf/ivysettings.xml 17/08/04 10:52:51 DEBUG InteractiveSession: InteractiveSession 9 app state changed from STARTING to FAILED 17/08/04 10:52:51 DEBUG InteractiveSession: InteractiveSession 9 session state change from shutting_down to dead 17/08/04 10:52:51 INFO InteractiveSession: Stopped InteractiveSession 9. 17/08/04 10:52:51 WARN InteractiveSession: (Fail to get rsc uri,java.util.concurrent.ExecutionException: java.io.IOException: Child process exited with code 1.)